Research Design Exam 2
What is discriminant validity?
Discriminate between groups
What is Split-Half reliability?
Dividing the testing group into two
What does the sloping line on a scatter plot indicate?
Perfect agreement
What is fence sitting responses?
Playing it safe by answering in the middle of the scale
What is a flashbulb memory?
Vivid memory of a particular instance (Ex: 9/11)
What is sample size referred to as in Psychology?
(N)
Best way to control the effects of question order?
Prepare different versions of a survey with the questions in different sequences
What are the two ways operational variables can be classified as?
1. Categorical: Male/Female/Non-Binary 2. Quantitative Variable: Height and Weight
What are three things that can threatened construct validity?
1. Observer bias - When observers see what they expect to see 2. Observer effects (or expectancy effects) - When participants confirm observer expectations (a) To prevent observer bias and observer effects - (Codebooks -clear rating instructions that are precise statements of how the variables are operationalized (b) Use militple observers (c) Even when an operationalization has good interrater reliability, it still might not be valid (d) Masked Research Design - (or blind design) in which the observers are inaware of the prupose of the study and the conditions to which participants have been assigned 3. Reactivity - When participants react to being watched (a) Blend in (obtrustive observations) - make yourself less noticeable) (b) Wait it out (get used to participants seeing your face and becoming comfortable with you) (c) Measure the Behavior's results - Use unobtrusive data - using indirect methods, researchers can measure behavior without doing any direct participant observation
What are the two cases where sampling error plays a role?
1. Probability samples, or true random samples 2. Convenience samples, where the probability cannot be chosen
What are the two types of naturalistic observation?
1. Purley naturalistic, where the observer remains completely removed from the sample 2. Participant, where the observer is involved in the solution
What are three common types of measures for variables?
1. Self-report - Operationalise a variable by recording people's answers to questions about themselves in a questionnaire or interview 2. Observational - Sometimes called a behavioral measure - operationalizes a variable by recording observable behaviors or physical traces of behaviors 3. Physiological - Operationalizes a variable by recording biological data
What three decisions must be made when doing a descriptive study?
1. Should the study be done in a laboratory, or in the neutral environment? 2. How are you going to collect data? 3. Should the observer be visible?
What are three examples of an Asking-Based Approach?
1. Survey research 2. Questionnaire research 3. Interviews
What is the value that r can ONLY fall between?
1.0 to -1.0
What is Cronbach's alpha?
A correlation-based statistic that measures a scale's interval reliability
What is strength?
A description of an association indicating how closely the data points in a scatter plot cluster along a line of best fit drawn through them
What is average inter-item correlation?
A measure of internal reliability for a set of items; it is the mean of all possible correlations computed between each item and the others
What is face validity?
A pleasurable operationalization of the variable in question (Ex: How big someone's head is - look at their head - instead of how intelligent someone is)
What is a sample in psychology?
A representative sample is a selected segment of a group that closely parallels the population as a whole
What is correlation coefficient R?
A single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between two variables
What is ecological function?
The role that various behaviors play in adapting to the environment
What is internal reliability (or internal consistency(?
A study participant giving a consistent pattern of answers
What is deviant-case analysis?
A type of case study that attempts to pinpoint how similar conditions can produce dramatically different outcomes
What is non-differentiation in responses?
A type of shortcut people take when answering survey questions (all strongly agrees or all 5's)
What are double-barreled questions?
Asks two questions in one
What is the advantage and disadvantage to simple random sampling?
ADVANTAGES: Best way to reduce sampling error by choosing from all members of the population DISADVANTAGES: Difficult to ensure that each member of a large population can be chosen in a sample
What are the advantages and disadvantages of cluster sampling?
ADVANTAGES: Makes it easier to choose from smaller clusters to adequately represent a population DISADVANTAGES: Possibly ignores segments of the population that are not in the clusters chosen from the sample
What are the advantages and disadvantages of stratified random sampling?
ADVANTAGES: Reduces bias that may simply be due to the characteristics of the population DISADVANTAGES: Can be difficult to ensure equal probability of being chosen from a large population using random sampling
What are the advantages and disadvantages to haphazard/ volunteer sampling?
ADVANTAGES: Very easy to obtain. Very common. DISADVANTAGES: Selection bias, so it poorly represents the sample (no random sampling)
What is reactivity in psychology?
An effect that occurs in descriptive research when the sample knows they are being observed
Ways to combat fence sitting responses?
Change the scale to have an even number of response options that way the responder is forced to choose a side because there is no neutral choice
Way to combat acquiescence?
Change the wording of some items to mean their opposite ("If I had my life to live over, I would change almost everything)
When the relationship of the coefficient r is weak what is it close to?
Close to 0
What is cluster sampling?
Clusters of individuals are identified and then a subset of clusters is randomly chosen to sample from
What is interrater reliability?
Consistent scores are obtained no matter who measures the variable
What did researcher's find about consumers?
Consumers' ratings were, instead, correlated with the cost of the product and the prestige of its brand
What is a case study?
Detailed examination of an individual or a small sample of individuals
When the relationship of the coefficient r is strong what is it close to?
Either 1 or -1
What are descriptive observations?
Enumerate what behaviors occur and in what quantity and frequency
What is quantitative measurements?
Enumerates or quantifies variables (numerical)
What is criterion validity?
Evaluated whether the measure under consideration is associated with a concrete behavioral outcome that it should be associated with according to the conceptual definition
Psychologists say that a particular measure is valid or not valid (T or F)
FALSE - Instead they ask what the weight of evidence is in favor of the measure's validity
What is longitudinal describe in observational studies?
Following one or ore sample for an extended period of time
What is Test-Retest reliability?
Giving the same test twice (parallel forms or alternate tests)
What is statistical reliability?
How likely one is able to obtain results by chance
What is Known-Groups Paradigm?
In which researchers see whether scores on the measure can discriminate among two or more groups whose behavior is already confirmed Ex: To validate the use of salivary cortisol as a measure of stress, a researcher could compare the salivary cortisol levels in two groups of people: those who are about to give a speech in front of a classroom and those who are in the audience. Public speaking is recognized as being a stressful situation for almost everyone. Therefore, if salivary cortisol is a valid measure of stress, people in the speech group should have higher levels of cortisol than those in the audience group
What is reactivity?
Is an effect that occurs when the knowledge that one is being observed alters the behavior being observed?
What is conceptual definition?
Is the researcher's definition of the variable in question at a theoretical level?
What is good about cross-sectional sampling?
It is inexpensive and quick
What is construct validity?
It makes sure it measures the conceptual variables it was intended for; Especially important when a construct is not directly observable
What is inter-observer reliability?
Measures where two or more observers yield similar results
What is the phenomena phycologists are interested in?
Motivation, emotion, thinking, reasoning (difficult to measure directly and hard to assess)
What are qualitative measurements?
Non-numerical "quality" of a variable, more descriptive (Ex: What kind of ice cream is your favorite?)
What is ordinal measurement?
Non-numerical; ordered (Ex: letter grades, education level)
What does qualitative mean?
Not numerical - ordinal and nominal
What does quantitative mean?
Numerical - interval and ratio
What is ratio measurement?
Numerical categories with a true zero point (Ex: reaction time, accuracy)
What is interval measurement?
Numerical categories without a true zero point; Quantitative data (Likert scale ratings 1 - 5)
What are cohort studies?
Observational studies that sample from a single group of individuals/ organisms
What is naturalistic observation?
Observing your sample in its natural environment
What do you always have to do with your data?
Operationalize
What is ethogram?
Relatively complete inventory of the specific behaviors performed by a member of a particular animal species
What are the 2 things that make a measurement good?
Reliability and Validity
What is operational definition?
Represents a researcher's specific decision about how to measure or manipulate the conceptual variable
What are the 2 things external validity concerns?
Samples and Settings
What is cross-sectional in psychology?
Sampling across a large group of individuals that vary along a single factor, like age or progression of illness
What are two statistical devices researchers can use for data analysis?
Scatterplots and the correlation coefficient r
Who is Sybil?
Shirley Ardell Mason (January 25, 1923 - February 26, 1998) was an American art teacher[1] who was reputed to have dissociative identity disorder (previously known as multiple personality disorder). Her life was purportedly described, with adaptations to protect her anonymity, in 1973 in the book Sybil, subtitled The True Story of a Woman Possessed by 16 Separate Personalities. Two films of the same name were made, one released in 1976 and the other in 2007. Both the book and the films used the name Sybil Isabel Dorsett to protect Mason's identity, though the 2007 remake stated Mason's name at its conclusion. Mason's diagnosis and treatment under Cornelia B. Wilbur have been criticized, with allegations that Wilbur manipulated or possibly misdiagnosed Mason.
What does it mean when data points cluster close to the sloping line of a scatter plot?
Strong interrater agreement
What is observer bias?
Systematic errors in observation that occur due to an observer's expectations
What is validity?
The appropriateness of a conclusion or decision
What happens when the N is large (i.e., the sample is large)?
The closer to the true population you get
What is reliability in science?
The consistency of a measure
What is sampling error?
The difference between the sample and the population
What is population in psychology?
The entire amount of people in a rendered geographical location
What is Haphazard/ Volunteer sampling?
The majority of psychological research studies. Members of population are chosen based on convenience and on who volunteers
What is content validity?
The measure contains all the parts it's supposed to contain (Ex: conceptual definition of intelligence - contains distinct elements, including the ability to "reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience" (Gottfredson, 1997, p. 13). To have adequate content validity, any operationalization of intelligence should include questions or items to assess each of these seven components)
What is ecological validity?
The parameters of a study must be approximate the natural setting as close as possible
What is ethology?
The study of naturally occurring behavior
What is slope direction?
The upward, downward, or neutral slope of the cluster of data points in a scatterplot
What is one problem with ethograms?
They can be highly subjective, meaning that the recording of the organism's behavior is dependent on the interpretation of the observer
What is operationalization?
Turing abstract concepts into measurable observations
What is case-control in psychology?
Two cohorts, which includes a more "typical" cohort ("control") and one cohort that contains a factor of interest ("case")
What is naturalistic observation?
Unobtrusive observations/ measures - Ability to observe without being observed
What is nominal measurement?
Unordered categories; Qualitative Data (Ex: University, city lived in, favorite ice cream, etc.)
What are interviewer methods?
When people are asked the questions directly by the researcher
What are self-report methods?
When people are asked to provide information about themselves in a questionnaire or survey?
What is semantic differential format?
When respondents might be asked to rate a target object using a numeric scale that is anchored with adjectives
What is the Likert Scale?
When such a sale contains more than one item and each response value is labeled with the specific terms "strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree"
Asking-Based Approach?
Where detailed information is gathered from a number of individuals
What is stratified random sampling?
Where the sample is constructed in such a way that the sampling reflects the TRUE VARIANCE in the population (Ex: If a university is 60% female, the sample should also have the same ratio of female participants)
What does external validity concern?
Whether the sample used in the study is adequate to represent the unstudied population
What is acquiescence?
Yeah-saying (yes or strongly agree to every item)
What are open-ended questions?
allow the respondents to answer in as much detail as they like
Ways to combat socially desirable responding?
include special survey items that identify socially desirable responders with target questions (i.e., My table manners at home are as good as when I eat out in a restaurant)
What is simple random sampling?
each member of a population is given an equal chance of being selected using random sampling
What are the advantages and disadvantages of self-report methods?
pros: easy, inexpensive can reach a wide audience cons: self-report bias - socially-desirable responding or "faking good" or "faking bad"
What are close-ended questions or forced-choice questions?
require respondents to select among a fixed number of option
