Research Methods Final

Content Validity

A measurement device's ability to be generalized to the entire content of what is being measured. ______ validity is based on comparing the content of the measure with the "universe" of content that defines the construct. For example, a measure of depression would have content that links to each of the symptoms that define the depression construct.

counterbalancing

A method of controlling for order effects in a repeated measures design by either including all orders of treatment presentation or randomly determining the order for each subject. Alternating the order in which participants perform in different conditions of an experiment. For example, group 1 does 'A' then 'B', group 2 does 'B' then 'A' this is to eliminate order effects.

probability sampling

A method used by pollsters to select a representative sample in which every individual in the population has an equal probability of being selected as a respondent. simple random sampling, stratified random sampling, cluster sampling problems: Feasibility: it is very difficult to identify all individuals in the population -ex: you cannot obtain a list of all depressed people - sampling frame is not always representative of the population (phone listings) - Even if the individuals in the population are identified, access may be limited and very expensive very important when you want to make precise state- ments about a specific population on the basis of the results of your survey.

Pearson correlation coefficient

A number between -1 and 1 that describes the linear relationship between pairs of measurement variables. Measures the degree to which two variables are related to one another

reactivity

A potential problem when measuring behavior is _______. A measure is said to be ______ if awareness of being measured changes an individual's behaviour. A reactive measure tells what the person is like when he or she is aware of being observed, but it doesn't tell how the person would behave under natural circumstances. Simply having various devices such as electrodes and blood pressure cuffs attached to your body may change the physiological responses being recorded. Knowing that a researcher is observing you or recording your behavior on tape might change the way you behave. Measures of behavior vary in terms of their potential reactivity. There are also ways to minimize reactivity, such as allowing time for individuals to become used to the presence of the observer or the recording equipment.

One-group pretest-posttest design

A quasi- experimental design in which the effect of an independent variable is inferred from the pretest-posttest difference in a single group. New teaching method in class A. Students in class A are measured at the beginning and end of the school year We measure a group of people, administer treatment, and measure the same group again ex: Class of students complete a math test, go through special math training, and are measured again at the end of the semester Advantages: There is a comparison point (as opposed to the case study) The people in the comparison "group" are the same as those in the treatment group - so there are no differences due to individual characteristics (reduction in random error) Disadvantages: A number of factors threat internal validity: History, Maturation, Testing, Regression to the mean

Nonequivalent control group pretest-posttest design

A quasi-experimental design in which nonequivalent groups are used, but a pretest allows assessment of equivalency and pretest- posttest changes. Ex: New teaching method in class A. Students in class A ,and students in class B are measured at beginning and end of the year No random assignment into groups One group receives a treatment, and another group does not - both groups are measured BEFORE and AFTER the treatment 2 groups of students (Israeli and international) are given a statistics quiz; Each learns with a different method; Both are tested again at the end of the year The closest it gets to a true experiment Selection might influence the change over time (even if the groups are equivalent on time 1)

Nonequivalent control group design

A quasi-experimental design in which nonequivalent groups of subjects participate in the different experimental groups, and there is no pretest. ex: New teaching method in class A. Students in class A and students in class B are measured at end of the school year No random assignment into groups One group receives a treatment, and another group does not - both groups are measured after the treatment Advantages: There is a comparison point (as opposed to the case study) History, testing, maturation, regression to the mean - are generally controlled for Disadvantages: Selection - The groups may differ because they had different people in them to begin with, regardless of treatment If participants might be different prior to the treatment - internal validity is seriously threatened

IMC

A question, similar in length and and response format, is embedded in the survey Unlike the other questions, the____ asks participants to ignore the standard response format and instead provide a confirmation that they have read the instructions (Instructions Manipulation Check) Each participant gets a score (passed vs failed)

random sampling

A sample that fairly represents a population because each member has an equal chance of inclusion protects external validity

quota sampling

A sampling procedure in which the sample is chosen to reflect the nu- merical composition of various subgroups in the population. A haphazard sampling technique is used to obtain the sample. A researcher who uses this technique chooses a sample that reflects the numerical composition of various subgroups in the population. Thus, quota sampling is similar to the stratified sampling procedure previously described; however, random sampling does not occur when you use quota sampling. To illustrate, suppose you want to ensure that your sample of students includes 19% freshmen, 23% sophomores, 26% juniors, 22% seniors, and 10% graduate stu- dents because these are the percentages of the classes in the total population. A quota sampling technique would make sure you have these percentages, but you would still collect your data using haphazard techniques. If you didn't get enough graduate students in front of the student union, perhaps you could go to a graduate class to complete the sample. Although quota sampling is a bit more sophisticated than haphazard sampling, the problem remains that no re- strictions are placed on how individuals in the various subgroups are chosen

pilot- study

A small-scale study conducted prior to conducting an actual experiment; designed to test and refine procedures.

regression analysis

A statistical procedure for determining the line that best fits the data by using all of the historical data points, not just the high and low data points.

purposive sample

A type of haphazard sample conducted to obtain predetermined types of individuals for the sample. The purpose is to obtain a sample of people who meet some predetermined criterion. Sometimes when I go to the movies, researchers will ask customers to fill out a questionnaire about one or more movies. They are always doing purposive sampling. Instead of sampling anyone walking toward the theater, they take a look at each person to make sure that they fit some criterion—under the age of 30 or an adult with one or more children, for example. This is a good way to limit your sample to a certain group of people. However, it is not a probability sample.

non-probability samples

Does not require the parameters of a population to be identified. No randomization. Often utilized in PT due to increased difficulty of meeting the more rigid requirements of probability sampling. - cheap and convenient -types: convenience sampling (haphazard sampling), snow-ball sampling, purposive sampling, quota sampling. -we don't know the probability of any particular member of the population being chosen. Type of sampling procedure in which one cannot specify the probability that any member of the population will be included in the sample.

concurrent validity

Groups that should theoretically differ on the measure show the predicted difference GRE - people high vs. low in IQ According to Social Dominance Theory, high status groups should be higher on SDO than low status groups

between

Hiring is an example of _____ subjects design (isolation).

order effect

In a repeated measures design, the effect that the order of introducing treatment has on the dependent variable. Potential solution: Counterbalancing

interviewer bias

Intentional or unintentional influence exerted by an interviewer in such a way that the actual or interpreted behavior of respondents is consistent with the interviewer's expectations.

filler items

Items included in a questionnaire measure to help disguise the true purpose of the measure.

within

Jurors decisions is an example of ______ subjects design (together)

cronbach's alpha

The most popular measure of reliability You administer the measure to one group of people You calculate the correlation between each 2 items The average of these correlations is your reliability An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question.

Cluster Sampling

The population is divided into groups (clusters) and then some groups are randomly selected to participate in the study. All people in the selected clusters are included in the sample. You divide the population into smaller clusters (100 classes) Your randomly select 20 clusters All students in those clusters are included in your sample

Snow-ball sampling

The researcher asks few participants to complete the survey and to disseminate it to other people s/he knows. Popular method with populations that are difficult to access Study in the Palestinian authority Subjected to different biases (people with friends, people who agree to participate)

convenience sampling

The researcher samples people from a population that is accessible to him/her. This accessible population is the sampling frame exs: People who suffer from depression at a certain clinic Study on objectification; Students from the IDC in terms of external validity, the question is whether the sampling frame is representative of the population of interest

predictive validity

The success with which a test predicts the behavior it is designed to predict; it is assessed by computing the correlation between test scores and the criterion behavior. Your measure predicts future behavior that it should predict theoretically A GRE test should predict grades in graduate programs

construct

Threats to ______ Validity: 1. When the operational definition is too narrow 2. When your operational definition can be interpreted in several ways, not necessarily matching the theoretical IV E.g., You wish to examine whether happiness leads the help giving In one condition participants are receiving gifts and in another condition they do not receive the gifts. Help giving in measured. Perhaps you are testing the effects of gratitude and not of happiness, on help giving - threat to construct validity

1

What size of correlation is considered "high" for convergent validity and what size is considered "low" for discriminant validity? No definite answer The correlations that constitute the convergence validity should be higher than those constituting discriminant validity If you measure correlations with other operationalizations of the same construct they should be quite high (r = 0.7 to 0.9) If you measure correlations with measures of related constructs the correlation should be medium in size (r=0.4 to 0.7)

double-blinded experiment

When both the participant and the experimenter do not know which condition they are in

Experimenter-related factors

When experimenters act in a way that drives participants to behave according to predictions - So it is not the treatment that caused the effect. If experimenters are aware of the purpose of the study, they might develop expectations of how the participants should respond Research has shown that participants pick up on experimenter's cues and might act accordingly

single blinded experiment

When only the participant does not know which condition s/he is in

confounding variable

When we actually know that an uncontrolled third variable is operating, we can call the third variable a confounding variable. If two variables are confounded, they are in- tertwined so you cannot determine which of the variables is operating in a given situation. any variable that is held constant cannot be a confounding variable

experimental control

With experimental control, all extraneous vari- ables are kept constant. If a variable is held constant, it cannot be responsible for the results of the experiment. In other words, any variable that is held constant cannot be a confounding variable.

convergence validity

Your measure correlates with other measures that assess the same theoretical construct A GRE test correlate positively with other intelligence measures If no other scales exist, the correlation between the new scale and measures of related constructs is assessed two measures believed to represent the same underlying construct will produce the same results

Panel study

a powerful type of longitudinal study in which a researcher observes exactly the same people, group, or organization across many time points. In a "two-wave" panel study, people are surveyed at two points in time; in a "three- wave" panel study, there are three surveys; and so on. Panel studies are particu- larly important when the research question addresses the relationship between one variable at "time one" and another variable at some later "time two.

double-barreled

a question that is really two questions in one. "Should senior citizens be given more money for recreation centers and food assistance programs?"

abstract

a summary of the research report and typically runs no more than 120 words in length. It includes information about the hypothesis, the procedure, and the broad pattern of results. Generally, little information is abstracted from the discussion section of the paper.

main threats

main threats to internal validity in repeated measures design: Order effects : Due to the order of treatments, participants respond differently Demand characteristics Potential solution: Counterbalancing

Close-ended measures

measures typically involve an "answering scale" that people respond to by choosing one number on the scale Likert-scale (most popular) limited number of response alternatives are given more structured approach they are easier to code and the response alternatives are the same for everyone more likely to be used when the dimensions of the variables are well defined.

Cronbach's alpha

nternal consistency indi- cator of reliability; correlation of each item on test with every other item based on the individual items; the researcher calculates the correlation of each item with every other item. A large number of correlation coefficients are produced; you would only want to do this with a computer! The value of alpha is based on the average of all the in-teritem correlation coefficients and the number of items in the measure. - more items will be associated with higher reliability. -very informative because they provide information about each individual item.

overcoming threats to internal validity

overcoming threats to internal validity: The study must involve a control group that is equivalent to treatment group in all aspects BUT the treatment People need to be randomly assigned to treatment and control groups Not always possible Even if you have a true experiment, you might still have problems of internal validity :

Contrast effect

participants react more strongly to the second measure because they compare it to the first. Evaluation of a person's characteristics that is affected by comparisons with other people recently encountered who rank higher or lower on the same characteristics

Systematic observation

refers to the careful observation of one or more specific behaviors in a particular setting. This research approach is much less global than naturalistic observation research. The researcher is interested in only a few very specific behaviors, the observations are quantifiable, and the researcher frequently has developed prior hypotheses about the behaviors.

Reliability

refers to the consistency or stability of a measure of behavior. a reliable measure of a psychological variable such as intelligence will yield the same result each time you administer the intelligence test to the same person. The test would be unreliable if it measured the same person as av- erage one week, low the next, and bright the next. Put simply, a reliable measure does not fluctuate from one reading to the next. If the measure does fluctuate, there is error in the measurement device. Any measure that you make can be thought of as comprising two components: (1) a true score, which is the real score on the variable, and (2) measurement error. An unreliable measure of intelligence contains considerable measurement error and so does not provide an accurate indication of an individual's true intelligence. In contrast, a reliable measure of intelligence— one that contains little measurement error—will yield an identical (or nearly identical) intelligence score each time the same individual is measured. When conducting research, you can measure each person only once; you can't give the measure 50 or 100 times to discover a true score. Thus, it is very important that you use a reliable measure. Your single administration of the measure should closely reflect the person's true score. tells us about measurement error but it does not tell us about whether we have a good measure of the variable of interest.

survey

research that employs questionnaires and interviews to ask people to provide information about themselves—their attitudes and beliefs, demo- graphics (age, gender, income, marital status, and so on) and other facts, and past or intended future behaviours. an important way for researchers to study relationships among variables and ways that attitudes and behaviors change over time. also important as a complement to experimental research findings.

program evaluation

research that evaluates the social reforms and innovations that occur in government, education, the criminal justice system, industry, health care, and mental health institutions.

Availability heuristic

tendency to judge the likelihood of events by availability in memory ; if instances come readily to mind, we presume such events are common Which of the following causes more deaths in the United States? Stomach cancer Motor vehicle accidents (Stomach cancer is far more common cause of death

Regression to the mean

threat internal validity: Extreme scores on first measurement tend to regress to the mean on the second measurement A phenomenon that occurs because of error in measurement If the score turned out to be extreme, it could be true, but it could also be due to random error due to problems with the measure (by chance you got an extreme number). Because your extreme score could be due to chance, next time you measure it you are more likely to get a score that is less extreme. due to statistical issues and not due to manipulation etc. EX: You invented a pill that you think can make people smarter. You decide to test it on extremely low-intelligent people because they're the ones who need it most. You give people an IQ test, select those 10% with the worst scores, and give them the drug. Then you re-test these 10% afterwards. You found the IQ of those low-intelligent group improved This could be due to regression to the mean The scores on the IQ test is a function of true score, and some chance component (error). Most people should score around the mean, with less people having extreme scores. By selecting the bottom 10% of scores, you're selecting people who might have gotten there by chance (e.g., one person was extremely tired that day, and so scored really low). These people are likely to score higher the next time you measure them.

History

threat to internal validity: Any external event can be responsible for the results Class example: A political event, a media event, changes in whether, etc. If little time lapsed between measurements there is less concern about history

Stratified random sampling

to ensure equal representation of different segments of the population, the population is divided into sub-groups (e.g., by race/ethnicity), and from each, a random sample is taken Proportions in sample and population are the same You obtain a list of all students (sampling frame) You create sub-groups based on specific schools From each school you randomly select the proportional number of people Your sample has 1000 people. if psychology has 20% of students, you randomly select 200 people from psychology

moderator

variables that affects the strength of the relationship between the IV and the DV Answers the WHEN question sets the boundaries of the effect A variable that influ- ences the nature of the relationship between two other variables (an independent variable and a dependent variable). In a factorial design, the effect of the moderator variable is revealed as an interaction.

self-report measures

A method of gathering data about people by asking them questions about a sample of their behavior. Anytime a person reports on his/her attitudes, feelings, judgments, impressions intentions to behave

total-item correlation

A method to increase reliability. Measure the correlation between each item and the average score of the scale and exclude items with weak correlation to total score.

Sampling frame

A list that includes all members of the population, from which a sample is drawn

true

A measure can be reliable, but not valid It will give you the same score every time you use it, but it won't measure your theoretical construct It measures something with no error, but it is not your theoretical construct

questions in surveys should:

be simple, not be double-barrelled or loaded, should not contain negative or

Curvilinear Relationship

increases in the values of one variable are accompanied by both increases and decreases in the values of the other variable. sometimes referred to as a nonmonotonic function. This particular relationship is called an inverted-U relationship the nature of the relationship between the dependent variable and independent variable changes at certain levels of the variables. Ex. Anxiety increases as a patient spends more days in hospice, until he has passed the 4th day, at which point, anxiety begins to decrease. Ex. smoking and education. Increases in visual complexity are accompanied by increases in liking for the stimulus, but only up to a point. The relationship then becomes negative, as further increases in complexity are accompanied by decreases in liking for the stimulus.

Randomization

Controlling for the effects of extraneous variables by ensuring that the variables operate in a manner determined entirely by chance.

construct validity

Degree to which a test actually measures what it claims to measure Extent to which a selection device measures the theoretical construct or trait (e.g., intelligence or mechanical comprehension).

face validity

Form of validity that is inferred by subjectively assessing whether the predictor "looks as if" it is related to job performance & has clearly similar content Establishes how well the assessment appears to measure it purpose.

Scientific Approach

Goal: Discover general laws that would allow to explain, predict and control phenomenon observation-theory- prediction-verification (methods, stats)- revision

negative case analysis

In field observation, the researcher must repeatedly sort through the data to develop hypotheses to explain the data, and then make sure all data are consistent with the hypotheses. An examination of observations that do not fit with the explanatory structure devised by the researcher is called a ______ ____ _____. In this case, the researcher revises the hypothesis and again examines all the data to make sure that they are consistent with the new hypothesis. The researcher may even collect additional data in order to examine more closely the circumstances that led to the negative case.

naturalistic observation

Observing and recording behavior in naturally occurring situations without trying to manipulate and control the situation. the researcher makes observations in a particular natural setting (the field) over an extended period of time, using a variety of techniques to collect information. A researcher uses _______ ______ when he or she wants to describe and understand how people in a social or cultural setting live, work, and experience the setting. researcher observes everything—the setting, the patterns of personal relationships, people's reactions to events, and so on. Concealed observation may be preferable because the presence of the observer may influence and alter the behavior of those being observed. Still, nonconcealed observation may be preferable from an ethical viewpoint; Consider the invasion of privacy when researchers hid under beds in dormitory rooms to discover what college students talk about. people often quickly become used to the observer and behave naturally in the observer's presence.

temporal precedence

Part of causal inference; the cause precedes the effect in a time sequence. we need to know that television viewing occurred first and aggression then followed

Self-serving bias

Rationalizing or thinking about things in a way that reflects positively on the self Exs: Success - I am a genius Failure - The test was WAY too difficult, The tendency for individuals to attribute their own successes to internal factors while putting the blame for failures on external factors.

Relationships between variables

Remember that these are general patterns. Even if, in general, a positive linear relationship exists, it does not necessarily mean that everyone who scores high on one variable will also score high on the second variable. Individual devi- ations from the general pattern are likely. In addition to knowing the general type of relationship between two variables, it is also necessary to know the strength of the relationship.

predictive validity

Research that uses a measure to predict some future behavior is using the _____ ______ approach. Thus, the criterion is some future behavior. GRE, LSAT The construct validity of such measures is demonstrated when scores on the measure predict the future behaviors. For example, predictive validity of the LSAT is demonstrated when research shows that people who score high on the test do better in law school than people who score low on the test. The measure can be used to advise people on whether they are likely to succeed in law school or to select applicants for law school admission.

Statistical Records

Statistical records are collected by many public and private organizations. The U.S. Census Bureau maintains the most extensive set of statistical records available to researchers for analysis. There are also numerous less obvious ones, including public health statistics and test score records kept by testing organiza- tions such as the Educational Testing Service. The sport of baseball is known for the extensive records that are kept on virtually every aspect of every game ever played. These statistics are available to anyone who wants to examine them. Reifman, Larrick, and Fein (1991) used two sources of archival data, baseball statistics and daily temperature, to study the relationship between heat and aggression. They examined all daytime major league games played in outdoor stadiums over a 3-year period to record the number of batters hit by a pitch. Hitting the batter is the measure of aggression. They also looked at the temperature on each day of the games. The results showed a clear link between temperature and aggression. More batters were hit on days when the temperature was above 90° (Fahrenheit) than when it was in the 80s. The number was lower still when temperatures were below 80°.

Test - Retest Reliability

Take measure two times Correlation of score at time one with score at time two; scores should be similar

illusory correlation

The perception of a relationship where none exists

causation

To conclude ________, three things must occur: 1. temporal precedence. 2. covariation of the cause and effect. 3. elimination of alternative explanations

Threat to autonomy

When minors are asked to participate, for example, a written consent form signed by a parent or guardian is generally required in addition to agreement by the minor; this agreement by a minor is formally called assent. Coercion is another threat to autonomy. Any procedure that limits an individ- ual's freedom to consent is potentially coercive. For example, a supervisor who asks employees to fill out a survey during a staff meeting or a professor requiring students in a class to participate in a study in order to pass the course is applying considerable pressure on potential participants. The employees may believe that the supervisor will somehow punish them if they do not participate; they also risk embarrassment if they refuse in front of co-workers. Sometimes benefits are so great that they become coercive. For example, a prisoner may believe that increased privileges or even a favorable parole decision may result from participation. Researchers must consider these issues and make sure that autonomy is preserved.

covariation of the cause and effect

When the cause is present, the effect occurs; when the cause is not present, the effect does not occur.

Archival research

involves using previously compiled information to answer research questions. the researcher does not actually collect the original data. Instead, he or she analyzes existing data such as statistics that are part of public records (e.g., number of divorce petitions filed), reports of anthropologists, the content of letters to the editor, or information contained in computer databases. three types of research data: statistical records, survey archives, and written records. issues: desired records may be difficult to obtain: They may be placed in long-forgotten storage places, or they may have been destroyed & we can never be completely sure of the accuracy of information collected by someone else.

Internal consistency reliability

is the assessment of reliability using re- sponses at only one point in time.

Scientific skepticism

means that ideas must be evaluated on the basis of careful logic and results from scientific investigations.

validity

refers to "truth" and the accurate representation of information Actually measuring exactly what you intend to measure

External Validity

the extent to which the results can be gener- alized to other populations and settings. Can the results be replicated with other operational definitions of the variables, with different participants, in other settings? concerns over the artificiality of laboratory experiments were addressed; this is an issue of _____ validity. Note that the goal of high internal validity may sometimes conflict with the goal of external validity.

field experiment

the independent variable is manipulated in a natural setting. the researcher attempts to control extraneous variables via either randomization or experimental control. The advantage is that the independent variable is investigated in a natural context. The disadvantage is that the researcher loses the ability to directly control many aspects of the situation. aka naturalistic observation or field observation

beneficence

the need for research to maximize benefits and minimize any possible harmful effects of participation we must calculate potential risks and benefits that are likely to result; this is called a risk- benefit analysis. Ethical principles require asking whether the research procedures have minimized risk to participants

Confirmation bias

the tendency to seek information that supports our schemas , A tendency to search for information that confirms one's preconceptions

peer review

, A process in which independent scientific experts read scientific studies before their publication to ensure that the authors have appropriately designed and interpreted their study. Before a study is published in a scientific publication, it must be reviewed by peers, other scientists who have the expertise to carefully evaluate the research and recommend whether the research should be published.

nonreactive and unobtrusive

A book by Webb, Campbell, Schwartz, Sechrest, and Grove (1981) has drawn attention to a number of measures that are called nonreactive or unobtrusive. Many such measures involve clever ways of indirectly recording a variable. For example, an unobtrusive measure of preferences for paintings in an art museum is the frequency with which tiles around each painting must be replaced—the most popular paintings are the ones with the most tile wear. Levine (1990) studied the pace of life in cities, using indirect measures such as the accuracy of bank clocks and the speed of processing standard requests at post offices to measure pace of life. Some of the measures described by Webb et al. (1981) are simply humorous. For instance, in 1872, Sir Francis Galton studied the efficacy of prayer in producing long life. Galton wondered whether British royalty, who were frequently the recipients of prayers by the populace, lived longer than other people. He checked death records and found that members of royal families actually led shorter lives than other people, such as men of literature and science. The book by Webb and his colleagues is a rich source of such nonreactive measures. More important, it draws attention to the problem of reactivity and sensitizes researchers to the need to reduce reactivity whenever possible. We will return to this issue at several points in this book.

reliability coefficient

A correlation that reflects the consistency and stability of a measure, over time, across equivalent forms, or among internal items.

simulations

A different type of role-playing involves simulation of a real-world situation. Simulations can be used to examine conflict between competing individuals, driving behavior using driving simulators, or jury deliberations, for example. Such simulations can create high degrees of involvement among participants. Even with simulations, there may be ethical problems. A dramatic example is the Stanford Prison Experiment conducted by Zimbardo

intuition

A problem with intuition is that numerous cognitive and motivational biases affect our perceptions, and so we may draw erroneous conclusions about cause and effect. a cognitive bias: illusory correlation you accept unquestioningly what your own personal judgment or a single story about one person's experience tells you about the world. often, involves finding an explanation for our own behaviors or the behaviors of others or is used to explain intriguing events that you observe illusory correlations are also likely to occur when we are highly motivated to believe in the causal relationship

Negative Linear Relationship

A relationship in which increases in the values of the first variable are accompanied by decreases in the values of the second variable.

Positive Linear Relationship

A relationship in which increases in the values of the first variable are accompanied by increases in the values of the second variable.

Belmont report

A report outlining three major principles (respect for persons/ autonomy, beneficence, and justice) foundational for the conduct of ethical research with human subjects

minimal risk

A second type of research activity is called _____ ____; means that the risks of harm to participants are no greater than risks encountered in daily life or in routine physical or psychological tests. When conducted, elaborate safeguards are less of a concern, and approval by the IRB is routine. Fully informed consent generally not required, but debriefing/ethical concerns are important. Some of the research activities considered minimal risk are (1) recording routine physiological data from adult participants (e.g., weighing, tests of sensory acuity, electrocardiography, electroencephalography, diagnostic echography, and voice recordings); note that this would not include recordings that might involve invasion of privacy; (2) moderate exercise by healthy volunteers; (3) research on individual or group behavior or characteristics of individuals, such as studies of perception, cognition, game theory, or test development in which the researcher does not manipulate participants' behavior and the research will not involve stress to participants. Studies of cognition/ perception not involving stress; Voice recordings not involving danger to participants; Standard psychological measures.

coding system

A set of instructions or rules on how to systematically observe and record content. a set of rules; should be as simple as possible, allowing observers to easily categorize behaviors.

Content analysis

A technique that turns qualatitive data into Quantitive data. analysis requires researchers to devise coding systems that raters can use to quantify the information in the documents.

debriefing

A verbal description of the true nature and purpose of a study and tell participants what kinds of results are expected; the practical implications of the results may also be discussed occurs after the completion of the study. It is an opportunity for the researcher to deal with issues of withholding information, deception, and potential harmful effects of participation.

measurement error

An error that occurs when there is a difference between the information desired by the researcher and the information provided by the measurement process. The _______ _____ in the unreliable test is revealed in the greater variability shown by the person who took the unreliable test.

case study

An observation technique in which one person is studied in depth in the hope of revealing universal principles. usually done when an individual possesses a particularly rare, unusual, or noteworthy condition.

Convergent Validity

Any given measure is a particular operational definition of the variable being measured. Often there will be other operational definitions—other measures—of the same or similar constructs. ______ ______ the extent to which scores on the measure in question are related to scores on other measures of the same construct or similar constructs. Measures of similar constructs should "converge"—for example, one measure of shyness should correlate highly with another shyness measure or a measure of a similar construct such as social anxiety.

Greater than minimal risk

Any research procedure that places participants at greater than minimal risk is subject to thorough review by the IRB. Complete informed consent and other safeguards may be required before approval is granted. Research involving physical stress, psychological stress, invasion of privacy, measures of sensitive information where participants may be identified Researchers planning to conduct an investigation are required to submit an application to the IRB. The application requires description of risks and benefits, procedures for minimizing risk, the exact wording of the informed consent form, how participants will be debriefed, and procedures for maintaining confidentiality. Even after a project is approved, there is continuing review. If it is a long-term project, it will be reviewed at least once each year. If there are any changes in procedures, researchers are required to obtain approval from the IRB.

empiricism

Assumption of Science. Information is collected by objective observations. Use of objective observations to answer a question about the nature of behavior.

negative wording

Avoid phrasing questions with negatives. This ques- tion is phrased negatively: "Do you feel that the city should not approve the pro- posed women's shelter?"

Theory

Broad statement A general law of behavior Many different ideas can be deduced from it We attempt to answer the research question by developing a general explanation 1. Organize and explain a variety of specific facts or descriptions of behavior. Such facts and descriptions are not very meaningful by themselves, and so theories are needed to impose a framework on them. This framework makes the world more comprehensible by providing a few abstract concepts around which we can organize and explain a variety of behaviors. 2. generate new knowledge by focusing our thinking so that we notice new aspects of behavior—theories guide our observations of the world. The theory generates hypotheses about behavior, and the researcher conducts studies to test the hypotheses. If the studies confirm the hypotheses, the theory is supported. As more and more evidence accumulates that is consistent with the theory, we become more confident that the theory is correct.

Operational definition

Examples of : aggression could be defined as (1) the number and duration of shocks delivered to another person, (2) the number of times a child punches an inflated toy clown, (3) the number of times a child fights with other children during recess, (4) homicide statistics gathered from police records, (5) a score on a personality measure of aggressiveness, or even (6) the number of times a batter is hit with a pitch during baseball games

response set

a tendency to respond to all questions from a par- ticular perspective rather than to provide answers that are directly related to the questions. most common _____ ____ is called social desirability, or "faking good."- leads the individual to answer in the most socially acceptable way—the way that "most people" are perceived to respond or the way that would reflect most favorably on the person.

face validity

It looks as if it measures what it is supposed to measure. The simplest way to argue that a measure is valid is to suggest that the measure appears to accurately assess the intended variable. the evidence for validity is that the measure appears "on the face of it" to measure what it is supposed to measure. not very sophisticated; it involves only a judgment of whether, given the theoretical definition of the variable, the content of the measure appears to actually measure the variable. That is, do the procedures used to measure the variable appear to be an accurate operational definition of the theoretical variable? a very subjective, intuitive process

Split-half reliability

One indicator of internal consistency; this is the correlation of an individual's total score on one half of the test with the total score on the other half. created by randomly dividing the items into two parts.

yea-saying

Participant response set to agree with all questions asked

falsifiability

Popper's (1959) assertion that the scientific method is ultimately based on our ability to prove an assertion is false (by finding a counterexample to it), but not to prove one is true. , Claims must be capable of being disproved.

Basic Research

Pure science that aims to increase the scientific knowledge base. tries to answer fundamental questions about the nature of behavior. often designed to address theoretical issues concerning phenomena such as cognition, emotion, motivation, learning, psychobiology, personality development, and social behavior

no risk

Research in which there is ___ ____ is exempt from review. Thus, research that involves only anonymous questionnaires, surveys, and educational tests is exempt, as is naturalistic observation in public places when there is no threat to anonymity. Archival research in which the data being studied are publicly available or the participants cannot be identified is exempt as well. This type of research requires no informed consent. However, there must be an institutional mechanism to determine that the research is in fact exempt. Researchers cannot decide by themselves that research is exempt; instead, the IRB at the institution formulates a procedure to allow a researcher to apply for exempt status. Studying normal educational practices; Cognitive aptitude/ achievement measures; Anonymous surveys; Observation of non- sensitive public behaviors where participants cannot be identified

applied research

Scientific study that aims to solve practical problems conducted to address issues in which there are prac- tical problems and potential solutions

psychobiography

a type of case study in which a researcher applies psychological theory to explain the life of an individual, usually an important historical figure.

multi-method approach

Testing a prediction in both experimental and no-experimental designs; this provides the researcher more confidence that the findings can generalize to more than one specific method or context

autonomy

The Belmont Report's principle of respect for persons or autonomy states that par- ticipants are treated as autonomous; they are capable of making deliberate deci- sions about whether to participate in research

third-variable problem

The concept that a correlation between two variables may stem from both being influenced by some third variable. any variable that is extraneous to the two variables being studied. When we actually know that an uncontrolled third variable is operating, we can call the third variable a confounding variable.

Interrater Reliability

The degree to which two or more independent observers agree in their ratings of another organism's behaviour.

Results

The researcher presents the findings, usually in three ways. 1. A description in narrative form—for example, "The location of items was most likely to be forgotten when the location was both highly memorable and an unusual place for the item to be stored." 2. Results are described in statistical language. 3. Materials are often depicted in tables and graphs.

justice

The third ethical principle defined in the Belmont Report: addresses issues of fairness in receiving the benefits of re- search as well as bearing the burdens of accepting risks. The history of medical research includes too many examples of high-risk research that was conducted with individuals selected because they were powerless and marginalized within the society. EX: poor African Americans in Alabama were not treated for syphilis in order to track the long-term effects of this disease. The justice principle requires researchers to address issues of equity. Any decisions to include or exclude certain people from a research study must be jus- tified on scientific grounds. Thus, if age, ethnicity, gender, or other criteria are used to select participants, there must be a scientific rationale.

naturalistic observation issues

Two related issues facing the researcher are whether to be a participant or nonparticipant in the social setting and whether to conceal his or her purposes from the other people in the setting A potential problem with participant observation, however, is that the observer may lose the objectivity necessary to conduct scientific observa- tion. Remaining objective may be especially difficult when the researcher already belongs to the group being studied (e.g., a researcher who belongs to Parents Without Partners and who undertakes a study of that group). Remember that naturalistic observation requires accurate description and objective interpretation with no prior hypotheses; If a researcher has some prior reason to either crit- icize people in the setting or give a glowing report of a particular group, the observations will likely be biased and the conclusions will lack objectivity. Concealed observation may be preferable because the presence of the observer may influence and alter the behavior of those being observed. concealed observation is less reactive than nonconcealed observation because people are not aware that their behaviors are being observed and recorded. Still, nonconcealed observation may be preferable from an ethical viewpoint: Consider the invasion of privacy when researchers hid under beds in dormitory rooms to discover what college students talk about.

discriminant validity

When the measure is not related to variables with which it should not be related, _______ ______ is demonstrated. The measure should discriminate between the construct being measured and other unrelated constructs. In research on the discriminant validity of their shyness measure, Bortnik et al. (2002) found no relationship between Shy Q scores and several conceptually unrelated interpersonal values such as valuing forcefulness with others.

No relationship

When there is ____ _______ between the two variables, the graph is simply a flat line. Unrelated variables vary independently of one another. Increases in crowding are not associated with any particular changes in performance; thus, a flat line describes the lack of relationship between the two variables.

true score

the amount of the observed score that truly represents what you are intending to measure; the real score on the variable.

role playing

alternative to deception: the experimenter describes a situation to participants and then asks them how they would respond to the situation not generally considered to be a satisfactory alternative to deception.One problem is that simply reading a description of a situation does not involve the participants very deeply—they are not part of a real situation. Also, because the experimenter gives the participants a complete description of the situation, the experimenter's hypothesis may become transparent to the participants. When people can figure out the hypothesis, they may try to behave in a way that is consistent with the hypothesis. Features of the experiment that may inform participants about the hypothesis are called "demand characteristics." The most serious defect is that, no matter what results are obtained, critics can always claim that the results would have been different if the participants had been in a real situation.

item-total correlations

the correlation between scores on individual items with the total score on all items of a measure. very informative because they provide information about each individual item.

Survey archives

consist of data from surveys that are stored on computers and available to researchers who wish to analyse them. Major polling organizations make many of their surveys available are extremely important because most researchers do not have the financial resources to conduct surveys of randomly selected national samples; the archives allow them to access such samples to test their ideas.

Theoretical definitions

defining variables in abstract or conceptual terms

Concurrent Validity

demonstrated by research that examines the relationship between the measure and a criterion behavior at the same time. Suppose you have a measure of shyness. Your theory of shyness might lead you to expect that salespeople whose job requires making "cold calls" to potential customers would score lower on the shyness measure than salespeople in positions in which potential customers must make the effort to contact the company themselves. Another approach to concurrent validity is to study how people who score either low or high on the measure behave in different situations. For example, you could ask people who score high versus low on the shyness scale to describe themselves to a stranger while you measure their level of anxiety. Here you would expect that the people who score high on the shyness scale would exhibit higher amounts of anxiety.

Method

divided into subsections, with the number of subsections determined by the author and dependent on the complexity of the research design. Sometimes, the first subsection presents an overview of the design to prepare the reader for the material that follows. The next subsection describes the characteristics of the participants. The next subsection details the procedure used in the study. In describing any stimulus materials presented to the participants, the way the behavior of the participants was recorded, and so on, it is important that no potentially crucial detail be omitted. Such detail allows the reader to know exactly how the study was conducted, and it provides other re- searchers with the information necessary to replicate the study. Other subsec- tions may be necessary to describe in detail any equipment or testing materials that were used.

written records

documents such as diaries and letters that have been preserved by historical societies, ethnographies of other cultures written by anthropolo- gists, and public documents as diverse as speeches by politicians or discussion board messages left by Internet users. Mass communication records include books, magazine articles, movies, television programs, and newspapers.

general laws/ theories

must be testable, parsimonious (simple), and involves a relationship between at least 2 variables (IV, DV).

informed consent

potential participants in a research project should be provided with all information that might influence their decision of whether to participate. Thus, research participants should be informed about the purposes of the study, the risks and benefits of participation, and their rights to refuse or terminate partic- ipation in the study. They can then freely consent or refuse to participate in the research. The content will typically cover: (1) the purpose of the research, (2) procedures that will be used including time involved (remember that you do not need to tell participants exactly what is being studied), (3) risks and benefits, (4) any compensation, (5) confidentiality, (6) assurance of volun- tary participation and permission to withdraw, and (7) contact information for questions.

loaded questions

question is written to lead people to respond in one way. "Do you favor eliminating the wasteful excesses in the public school budget?" and "Do you favor reducing the public school budget?" will likely elicit different answers. Or consider that men are less likely to say they have "raped" someone than that they have "forced sex";

Internal validity

refers to the ability to draw conclusions about causal relationships from our data. A study has high internal validity when strong inferences can be made that one variable caused changes in the other variable. We have seen that strong causal inferences can be made more easily when the experimental method is used. Internal validity is increased when the considerations of cause and effect that were previously discussed can be applied to the research.

construct validity

refers to the adequacy of the operational definition of variables: Does the operational definition of a variable actually reflect the true theoretical meaning of the variable? Because variables can be measured and manipulated in a variety of ways, there is never a perfect operational definition of a variable. Over time, many researchers will use multiple methods to operationally define any variable.

Discussion

the researcher reviews the research from various perspectives. Do the results support the hypothesis? If they do, the author should give all possible explanations for the results and discuss why one explanation is superior to another. If the hypothesis has not been supported, the author should suggest potential reasons. What might have been wrong with the methodology, the hypothesis, or both? The researcher may also discuss how the results compare with past research results on the topic. This section may also include suggestions for possible practical applications of the research and for future research on the topic.

hypothesis

type of idea or question; it makes a statement about something that may be true. Thus, it is a tentative idea or question that is wait- ing for evidence to support or refute it.

Interrupted time-series design

A design in which the effectiveness of a treatment is determined by examining a series of measurements made over an extended time period both before and after the treatment is introduced. The treatment is not introduced at a random point in time.

mixed factorial design

A design that includes both independent groups (between- subjects) and repeated measures (within- subjects) variables.

baseline

A condition of an experiment in which the independent variable is not present; data obtained during baseline are the basis for determining the effects of the independent variable; a control condition that does not necessarily mean the absence of instruction or treatment, only the absence of a specific independent variable of experimental interest.

cross-sectional method

A developmental research method in which persons of different ages are studied at only one point in time; conceptually similar to an independent groups design.

factorial design

A design in which all levels of each independent variable are combined with all levels of the other independent variables. A _____ _____ allows investigation of the separate main effects and interactions of two or more independent variables.

Within-subjects factor

An experimental design which uses the same participants in both the experimental and control groups The same participants are assigned to each level of the factor (IV) The error term is typically smaller compared with between-subjects designs economical Some research questions require a repeated measures design aka repeated measures design

confidence interval

An interval of values within which there is a given level of confidence (e.g., 95%) where the population value lies you can have 95% confidence that the true population value lies within this interval around the obtained sample result When researchers make inferences about populations, they do so with a certain degree of confidence The confidence interval gives you information about the likely amount of the error. The formal term for this error is sampling error, although you are probably more familiar with the term margin of error.

True score theory

Any variable we measure (observe) is composed of two factors: true score and error component Observed score = true score + measurement error EX: You developed a scale for weight Observed score = true weight + error in the scale If your measure is reliable, it has a small error component - the observed score reflects the true score The more reliable is your measure, the less error it captures If your measure is reliable, it will produce the same scores everytime it is used If your measure is unreliable, it will generate a different score everytime you use it

instrument decay

As a threat to internal validity, the possibility that a change in the characteristics of the measurement instrument is responsible for the results.

random assignment

Assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups protects internal validity

Simple random sampling

Consists of individuals from the population chosen in such a way that every set of individuals has an equal chance to be a part of the sample actually selected. Poor sampling methods can lead to misleading conclusions. Because no one is excluded, you can conclude that your findings apply to anyone from the population (including those you did not sample)

open-ended measures

Content is judged by coders Requires a coding scheme respondents are free to answer in any way they like. require time to categorize and code the responses and are therefore more costly. Sometimes a respondent's response cannot be categorized at all be- cause the response doesn't make sense or the person couldn't think of an answer. can yield valuable insights into what people are thinking most useful when the researcher needs to know what people are thinking and how they naturally view their world

construct

Determining the _____ validity of your IV: 1. Face validity - Subjective judgment 2. Manipulation check Based on your theory, develop questions that measure whether the construct was indeed manipulated TST (20 statements test) Power manipulation It is a good idea to add measures of constructs you did not wish to manipulate (e.g., embarrassment in body study) 3. Pre-test your manipulations 4. Rely on existing methods (validated)

between-subjects factor

Different participants are assigned to each level of the factor (IV) Between subject experiment with two conditions: different subjects are randomly allocated to the 2 conditions. Insures the groups will be equal in all respects, except by chance. This enables us to compare their performance between experimental and control conditions. Any difference can be attributed to differences between conditions

2

Disadvantages of Test-Retest Reliability: Practice effect - participants "remember" the scale and recall their answers. The correlation will be inflated Some phenomenon might change across time and therefore the test-retest approach is inappropriate for assessing consistency of a measure Often problematic to recruit the same people twice - anonymity issues

floor effect

Failure of a measure to detect a difference because it was too difficult

ceiling effect

Failure of a measure to detect a difference because it was too easy.

physiological measures

Measures of bodily responses, such as blood pressure or heart rate, used to determine changes in psychological state.

Reliability as Internal Consistency

Methods of assessing reliability by administrating the test at only one point in time Common for testing reliability of multi-item scales You assess the extent to which the different items are consistent with one another, and are measuring a single construct split-half reliability, cronbach's alpha, total-item correlation

Ecological validity

The extent to which a study is realistic or representative of real life. the degree to which the procedures in the study relate to real-life situations (field experiments are ecologically valid)

construct

Non-experimental designs are typically higher on ______ validity than experimental designs

External

Overcoming Threats to ______ Validity: Use a multi-method approach (laboratory studies, correlational studies, field experiments) Test your predictions among different samples Use a variety of measures and manipulations Different ways of operationalizing the same thing

demand

Overcoming ______ characteristics: 1. Cover story to disguise the exact purpose of the study Participants should not be able to figure out the predictions 2. Use implicit measures Word completion 3. Add filler tasks and filler items to disguise the purpose of the study Add another session for testing the DV (the 2 study paradigm) 4. Ask the participants specifically to be honest, and assure them that there is no right or wrong answer Can also present them with responses of "other people" to make sure they don't think cooperation is the norm

Experiment

Overcoming _______'s effects: Experimenter's should be well trained Practice the consistent behavior across everyone (script) Minimize the experimenter's involvement (recorded instructions, computerized studies) Keeping the experimenter unaware of true purpose and predictions Keeping the experimenter unaware of the condition the person is in E.g., The computer determines the condition Yale hat vs. Princeton hat study Double/single blind experiment

participant variable

Personal attributes of the participant that are considered as moderators E.g., Age, gender, ethnicity These IV's are always measured and not manipulated

behavioural measures

Personality assessments that involve observing an individual's behaviour in a controled situation

conceptual replication

Replication of re- search using different procedures for manipulating or measuring the variables.

quasi-experimental designs

Research method similar to an experimental design except that it makes use of naturally occurring groups rather than randomly assigning subjects to groups. suffer from threats to internal validity

haphazard sampling

Selecting subjects in a convenient manner, usually on the basis of availability, and not with regard to having a representative sample of the popula- tion; a type of nonprobability sampling. aka convenience sampling

SDO

Social dominance orientation: A personality variable measuring the degree to which a person supports inequality among social groups.

fatigue effect

Students might get bored, tired, as they go along Later materials will be less remembered due to fatigue a deterioration in performance as the research participant becomes tired, bored, or distracted Potential solution: Counterbalancing

One-group posttest-only design

Studies in which people are given a treatment and are then tested To be able to say SOMETHING about the treatment you must have a comparison point Case studies have no internal validity, and are of little, if any, scientific value Essentially, when there is no control, there is no comparison point ex: New teaching method is implemented in class A. Students are measured at end of the year

response rate

The _____ _____ in a survey is simply the percentage of people in the sample who actually completed the survey.

mediator

The ________ explains the mechanism behind the effect (why does it happen?) A variable that explains the relationship between the IV and the DV Answers the WHY question can be partial/full

interaction

The differing effect of one inde- pendent variable on the dependent variable, depending on the particular level of another independent variable.

naturalistic observation goals

The goal is to provide a complete and accurate picture rather than to test hypotheses formed prior to the study. To achieve this goal, the researcher must keep detailed field notes— that is, write or dictate on a regular basis (at least once each day) everything that has happened. Field researchers use a variety of techniques to gather informa- tion: observing people and events, interviewing key "informants" to provide in- side information, talking to people about their lives, and examining documents produced in the setting, such as newspapers, newsletters, or memos. In addition to taking detailed field notes, researchers conducting naturalistic observation usually use audio- and videotape recordings. The researcher's first goal is to describe the setting, events, and persons observed. The second, equally important goal is to analyze what was observed. The researcher must interpret what occurred, essentially generating hypotheses that help explain the data and make them understandable. A good naturalistic observation report will support the analysis by using multiple confirmations. For example, similar events may occur several times, similar information may be reported by two or more people, and several differ- ent events may occur that all support the same conclusion The data in naturalistic observation studies are primarily qualitative in nature; that is, they are the descriptions of the observations themselves rather than quantitative statistical summaries. Such qualitative descriptions are often richer and closer to the phenomenon being studied than are statistical representations. However, there is no reason that quantitative data cannot be gathered in a naturalistic observation study.

3

Using the IMC: The researcher assumes that those who fail the IMC fail to read other questions; IMC question must be similar in format to the rest of the questions Issues of generalizability: if those who failed the IMC differ from the rest of the sample in other aspects (e.g., gender, less intelligent pts) then the sample is biased and external validity is threatened Can be tested (No differences were found on age, gender, need for cognition, but other differences might still exist) When forced to continue those who failed the IMC showed the effect Issues of order: Beginning - Participants might be reminded this way to pay attention; but they might also get insulted and therefore bias their responses after the IMC End - you detect those who don't pay attention, but you are more likely to loose a large part of your sample

Test-Retest Reliability

We can test the same group of people twice, and examine the correlation among the measurement A method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions If people score similarity in both times (r larger than 0.7), it means that the scale is reliable Common method for assessing reliability of 1-item scales

Discriminant validity

Your measure does not correlate with other measures that assess different theoretical constructs The construct validity of a measure is assessed by examining the extent to which scores on the measure are not related to scores on conceptually unrelated measures. A GRE does not correlate with a test of motor skills Measures of constructs that theoretically should not be related to each other are, in fact, observed not to be related to each other here, stronger validity is reflected in a weak correlation

Reliability, validity

______ is a necessary but insufficient condition for ______. (reliability/validity)

Participant

_______ or subject variables are individual differences; these are the characteristics of individuals, including gender, intelligence, and personality traits such as extraversion

mediating

_______ variables are psychological processes that mediate the effects of a situational variable on a particular response

Response

_______ variables are the responses or behaviors of individuals, such as reaction time, performance on a cognitive task, and helping a victim in an emergency.

Situational

_______ variables describe characteristics of a situation or environment. The length of words that you read in a book, the spatial density of a classroom, the credibility of a person who is trying to persuade you, and the number of bystanders to an emergency

Operational definitions

a definition of the variable in terms of the operations or techniques the researcher uses to measure or manipulate it. allows us to observe the theoretical construct in reality For example, human intelligence may be defined as what an intelligence test measures. In order for variables to be studied empirically. also help us communicate our ideas to others. If someone wishes to tell me about aggression, I need to know exactly what is meant by this term because there are many ways of ______ _______ it.

Latin square

a formal system of partial counterbalancing that ensures that each condition appears in each position at least once Partial counterbalancing technique where the number of orders of conditions used is equal to the number of conditions in the study

reliability

consistency of a measure Are we measuring the true ability/characteristic that we wish to measure? How far are we from the true score? Do we capture a real score? Or just random variance? Ability of a test to yield very similar scores for the same individual over repeated testings

Demand characteristics

cues that inform the subject how he or she is expected to behave. Any feature of the experiment that might inform participants of the purpose of the study, and therefore bias their responses The assumption here is that the participant is motivated to cooperate with the demands of the study and might act in ways that would fit with the experimenter's expectations Therefore, it is not the treatment that caused the effect, but the attempts to fit with expectations Potential solutions: Time-intervals between treatments Good cover-story (a "second study") Implicit methods (IAT)

IRB

every institution that receives federal funds must have an Institutional Review Board (IRB) that is responsible for the review of research conducted within the institution. The IRB is a local review agency composed of at least five individuals; at least one member of the IRB must be from outside the institution. in addition, most psychology departments have their own research review committee

true experiment

experiment that enables to rule out alternative explanations to the effect Must involve random assignment to conditions and a proper control condition/s

practice effect

improvement in performance resulting from opportunities to perform a behavior repeatedly so that baseline measures can be obtained. Students might get better at the memory test as they go along Later materials will be remembered better due to better practice people gain experience and get better at the task (DV) and therefore respond differently Potential solution: Counterbalancing

Conclusion validity

is the extent to which the conclusions about the relation- ships among variables reached on the basis of the data are correct or what Trochim (2006) terms "reasonable." sometimes termed statistical conclusion validity; the concept focused on whether the sta- tistical conclusion about whether there is a relationship between variables is correct.

internal validity

the degree to which a cause and effect relationship can be established Refers to the research design Ask yourself: Based on this design, can I conclude a cause and effect? What else (external factor) might explain the results? More experimental control - less alternative explanations- stronger ______ _____ The main advantage of experimental designs over Non-experimental designs

External validity

the degree to which the findings can be generalized to other populations and settings Can the results be replicated with other operational definitions? With different participants? In other settings? A study can be internally valid (have a cause-and-effect relationship) but not externally valid (the effect is only true for the particular settings, procedures and participants)

Introduction

the researcher outlines the problem that has been investigated. Past research and theories relevant to the problem are described in detail. The specific expectations of the researcher are noted, often as formal hypotheses. In other words, the investigator introduces the research in a logical format that shows how past research and theory are connected to the current research prob- lem and the expected results. Always involves a funnel structure: Broad research problem Review of related previous research Proposed theory (general) Proposed hypothesis (more specific)

maturation

threat internal validity: Any change participants go through with the passage of time can explain the results Participants can grow older, hungrier, more tired, more bored The obtained effect can reflect the maturation process rather than the treatment

Testing

threat internal validity: First measurement impacted the performance on the second Participants are "primed" with the concept; are more motivated, more knowledgably about the purpose of the study Completing a sexism inventory might change the way people approach the issue the second time, regardless of the movie

Subject-related factors

threats to internal validity: When participants figure out what you are testing and answer accordingly - So it is not the treatment that caused the effect