PBSI 302: Exam 1 Review

experimenter bias

- "good subject" response - social desirability - "apprehensive subject" (evaluate apprehension) - Hawthorne effect - responses to experimenter expectancies

Causality

- "if football players wear gloves, they will catch more footballs"

Five levels of measurement (be able to identify the kind of variable [e.g., continuous] and scale of measurement [e.g., ordinal])

- 1. labels - 2. nominal scale - 3. ordinal scale - 4. interval scale - 5. ratio scale

statistical conclusion validity

- Appropriateness of inferences made from data as a function of conclusions drawn from statistical analyses - is the difference between groups (or relationship between the IV and DV) statistically significant? - is the difference / relationship simply a function of chance?

interval scale

- MOST COMMON LEVEL of measurement in psych! - measures how much of a variable or attribute is present - rank order of persons or objects is known with respect to attribute - how far apart is known - provides info about the magnitude - examples include how well do you like this course, where 1 represents no like and 5 represents love

conceptual definition

- abstract or general meaning of the construct - example in measuring anxiety: uneasiness and distress about future uncertainties

frequency

- amount - "there are 1400 psychology majors"

Which statistical test do I use to analyze my data?

- associations between variables or differences between means - how many variables or means/groups are there in the research design? - are the means / groups independent (between-subjects) or dependent (within-subjects)?

Characteristics of scientific disciplines/research: EMPIRICISM

- based on experience / observation / quantitative data - empiricism: approach of collecting and using quantitative data to develop, support, or challenge theories - theory: explanations of observed phenomena (why and how); permits predictions

validity

- concerns what it measures and how well it does so - it tells us what can be inferred from test scores - the ____ of a test cannot be reported in general terms - depends on the USE of the test - THE KEY CRITERION IN THE EVALUATION OF A TEST OR MEASURE

Characteristics and advantages of the scientific method

- control - use of operational definitions - empirical - objective - replication - self-correcting - progressive - tentative - parsimonious: Occam's razor = "one should not increase beyond what is necessary, the number of entities required to explain anything" - concerned with theory - "research" [journalist] vs. research

controls / solutions to construct validity threats

- double-blind procedures - single-blind procedures - deception - separate the IV and DV in time - use unobstructive methods / measures

criterion-related validity

- effectiveness of a test in predicting an individual's behavior in specific situations - the test or measure is intended as an indicator or predictor of some other behavior (that typically will not be observed until some future date) - concurrent, predictive, or postdictive

internal validity

- extent to which causal inferences or inferences about the observed relationships between variables are sound that is that the a. relationship between 2 variables or difference between 2 groups is causal in nature, or that the absence of a relationship implies the absence of cause or, b. the extent to which the observed relationship obtained from the research design / study is real or artifactual (not real / due to something else) - sound = set to be true or real - is the observed effect real or is it due to something else? - validity examines whether the study design, conduct, and analysis answer the research questions without bias.

construct validity

- extent to which labels placed on what is being observed are theoretically relevant - "do the results support the theory underlying the research" - the correct labels or theories to explain phenomena

nominal scale

- grouping objects or people without any specified quantitative relationship among the categories - examples: coding all men as 1 and women as 2 or cats as 1 and dogs as 2

ratio scale

- has properties of preceding 4 measurements in addition to true zero point - rank order of persons or objects is known in respect to attribute - how far apart is known - distance from a true zero point is known examples: speed (no motion)

threats to internal validity

- history - maturation - testing - mortality / attrition - selection - regression to the mean - instrumentation - noncompliance - diffusion of treatment PRIMARY CONTROL --> RANDOM ASSIGNMENT (randomization)

operational definition

- how the construct is measured or observed - more objective and specific the better - instrumentation of response - example is measuring anxiety: 1. self-report (5 items, Likert scale), 2. parent report (open-ended), 3. behavioral observation (5 minutes counting anxious behaviors), 4.physiological measure (pulse for 30 seconds)

external validity

- inference that results can be generalized to and across alternate measures, participants, settings, and times - how generalizable are the results? - examines whether the study findings can be generalized to other contexts. - are the findings unique to just the participants we studied or could they apply to other groups?

threats to external validity

- interaction between subjects and treatment: population validity - interaction between setting and treatment: ecological validity - interaction between history and treatment: temporal validity PRIMARY CONTROL --> RANDOM SAMPLING (for representativeness) - procedures used to ensure the sample is theoretically representative of the population

What entails a "good" theory (e.g., falsifiable, tentative)

- is it tentative, progressive, and self correcting / replicable - some knowledge is specific to a certain range of time (i.e. tentative) - collect more information (i.e. progressive) to... update knowledge (self-correction) or reaffirm knowledge (self-replication) - no single research study will discover truth - why we tend to be attentive to studies

threats to construct validity

- loose connection between theory and experiment - misalignment between conceptual and operational definition of a construct - e.g. competence measured with corporate success - diffusion of treatment

threats to statistical conclusion validity

- low statistical power, small sample size - violations of statistical assumptions - poor reliability of observed scores (measurement tools) - address threats --> adequate power, meet test assumptions, and use reliable measures

role demands

- participants' expectations of what the experiment requires them to do - can lead participants to engage in socially unacceptable behaviors

ordinal scale

- people or objects are ordered from "most" to "least" with respect to an attribute - there is no indication of "how much" is an absolute sense, any of the objects possess the attribute - there is no indication of how far apart the objects are with respect to the attribute - examples: college football pools, top 5 contestants in a beauty pageant

How many participants do I need for my study?

- power analysis / statistical power - sample size (n) - effect size (magnitude of effect) - power (.80) - alpha (p-value, .05)

Reliability

- refers to the presence of measurement error

association

- relationship - "children who play video games have shorter attention spans"

WHAT IS SCIENCE?

- science is a method of making sense of the world around us - a means of / approach to acquiring knowledge - numbers and data to base, no intuition, data driven

variable

- symbol that can assume a range of numerical values - some property of an organism / event that has been measured - an aspect of a testing condition that can change or take on different characteristics with different conditions - an attribute of a phenomena - something that we study! - some property of an object, phenomenon, or event whose measurement can take on two or more values! - can be called construct labels!

measurement

- the assignment of numbers to events/ objects according to rules that permit properties of the events / objects to be represented by properties of the number system - number assignment! - numbers reflect variations in the properties! - measurement systems can give you scores

construct-related validity

- the extent to which the test may be said to measure a theoretical construct or trait - construct: label for a theoretical dimension on which people are thought to differ - a construct represents a hypothesis sources of construct related validity evidence: convergent & divergent validity

content-related validity

- validity depends primarily on the adequacy with which a specified content domain is sampled - involves the degree to which a predictor covers a representative sample of the behavior being assessed

in class defintion of extraneous variable

- variable affecting a dependent variable when the variable is no interest to us - something you don't want

in class defintion of confound

- variable affecting development - influences non-separation - when independent changes, CV changes ex: DV = class discussion, IV = seats, confound = class size (to seating arrangement)

Confound

- when an extraneous variable systematically varies with variations or levels of the IV - A specific type of extraneous variable that systematically varies along with the variables under investigation and therefore provides an alternative explanation for the results. - When researchers design an experiment they need to ensure that they control for confounds; they need to ensure that extraneous variables don't become confounding variables because in order to make a causal conclusion they need to make sure alternative explanations for the results have been ruled out.

Labels

- when numbers are used as a way of keeping track of things without suggestion that the numbers can be subjected to math analyses - examples: participant ID, university identification number (UIN), and social security numbers

realism

belief or philosophy that perceptions exist outside the mind

For a measurement system or method to be any use in science, its scores must be...

both reliable and valid

example of content related validity

classroom tests

Laws vs. research questions vs. hypotheses vs. theories

laws: research questions: hypotheses: theories: general principle / set of principles re: why and how things occur - seeks to explain phenomena / the data suggests...

in class definition - scientific method

method of approaching to examining empirical data (testing hypotheses / propositions) and drawing conclusions about systematically observable phenomena

Characteristics of scientific disciplines/research: OBJECTIVITY

objective - perceived the same way - free from bias, opinion, preferences, values - subjective - unique to the perceiver - strives to be objective - not asserting its absolutely objective / relatively much more objective to knowledge acquisition

Characteristics of scientific disciplines/research: PARSIMONY

often the simplest explanation is the best explanation - parsimony: explain the relationship between X and Y - go w/ theories that use as little variables to explain phenomena

dependent variable

outcome, criterion, consequence

determinism / causality

phenomena are the result of some cause which precedes said phenomena (rejects concept of an accident / or things that just happen)

independent variable

predictor, precursor, antecedent

What is the most powerful control method in the scientific method?

random assignment! - allows us to deal with threats to research designs - designates research designs as experimental vs. non-experimental - not all studies use random assignment thus not all studies are experiments - experiments are a type of research study - while all experiments are studies, not all studies are experiments!

Working assumptions of science

realism, rationality, regularity, discoverability / causality, & discoverability

Rationality

reasoning and logic > authority, intuition, and faith

regularity

recurring patterns in phenomena of interest

translational research

research that uses knowledge derived from basic research to develop and test solutions to real-world problems

in class term - psychology behavioral science similarity

similar to hard science in the fact that psychologists are concerned with human behavior rather than the physical world

discoverbility

solutions / explanations exist for posed queries

alternate form reliability

temporal consistency or stability - and inder-form consistency or equivalence - a measure of the extent to which 2 separate forms of the same test are equivalent

test-retest reliability

temporal consistency or stability - involves the repeated administration of the same test to the same sample

Characteristics of scientific disciplines/research: CONTROL

the defining characteristic of scientific method! - control the alternate explanations!

scorer reliability or inter-rater reliability and agreement

the extent to which 2 or more raters are consistent, or agree - reliability = rank order - agreement = magnitude AND rank order

face validity

the extent to which a test or measure looks like it measures what its supposed to measure - it can influence test takers' reactions and attitudes toward the test

in class term - methods of acquiring knowledge

we consider the scientific method to be SUPERIOR to nonscientific methods

Basic research

Research conducted primarily for the sake of achieving a more detailed and accurate understanding of human behavior, without necessarily trying to address any particular practical problem.

applied research

Research conducted primarily to address some practical problem.

example of extraneous variable

Suppose researchers want to find out the best teaching method that will ensure that most students pass the math exam. An extraneous variable in this case will be the prior knowledge that students possess related to the same math exam that can affect the present score.

extraneous variable

any variable other than the independent variable that influences the dependent variable

random assignment

assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups

Questionable research practices (QRPs)

1. The selective deletion of outliers in order to influence (usually by artificially inflating) statistical relationships among the measured variables. 2. The selective reporting of results, cherry-picking only those findings that support one's hypotheses. 3. Mining the data without an a priori hypothesis, only to claim that a statistically significant result had been originally predicted, a practice referred to as "HARKing" or hypothesizing after the results are known 4. A practice colloquially known as "p-hacking" in which a researcher might perform inferential statistical calculations to see if a result was significant before deciding whether to recruit additional participants and collect more data. As you have learned, the probability of finding a statistically significant result is influenced by the number of participants in the study. 5. Outright fabrication of data although this would be a case of fraud rather than a "research practice."

Definition of the four evidential bases of test validity

1. criterion-related validity 2. content-related validity 3. construct-related validity 4. face validity

four evidential bases of research validity

1. internal validity 2. external validity 3. statistical-conclusion validity 4. construct validity

Pitfalls of approaches/methods of acquiring knowledge

1. science - one of the several ways or methods of acquiring knowledge about behavior. seeks to develop theoretical and empirical explanations of behavioral phenomena. - science is a method of inquiry; it is the application of logic to empirical evidence 2. other scientific methods - (some discussed in textbook) A. tenacity: a method of acquiring knowledge based on superstition or habit (e.g. "Old dogs can't learn new tricks" - but elderly can and do learn; "Spare the rod, spoil the child") - mere exposure - development of positive attitude toward something as a function of increased familiarity with it (e.g. political ads and internet banners) B. common sense: "practical intelligence" shared by a large group of persons (e.g. fire, initial, common sense response is to douse with water - however, the effectiveness of this response is a function of the source of combustion; several instances when this would be a very inappropriate response) C. intuition: spontaneous perception or judgement not based on rational or logical step (e.g., psychics) D. mysticism: belief in insight gained by means of a private experience such as an altered state or consciousness (e.g. hallucinations) E. authority: acceptance of information because it is acquired from a highly respected, credible, or popular source (e.g. physician recommendation of aspirin, rotten tomatoes movie recs)

determinants of internal validity

1. standardization of research design and variables; sound research design practices 2. quality of the research design: controlling extraneous and confounding variables

Definition of the four predominant forms of reliability evidence

1. test-retest reliability 2. alternate-form reliability 3. split-half, odd-even (or random split) reliability 4. coefficient alpha (Cronbach's alpha) 5. scorer reliability or inter-rater reliability and agreement

example of dependent variable

1. the effect of rater personality on supervisor ratings of performance 2. How does color influence your mood? 3. 65 vs 75 vs 85 years old: Who is the happiest? 4. The effectiveness of tutoring for 8th grade students 5. Violence in the workplace: videogames or access to weapons?

example of independent variable

1. the effect of rater personality on supervisor ratings of performance 2. How does color influence your mood? 3. 65 vs 75 vs 85 years old: Who is the happiest? 4. The effectiveness of tutoring for 8th grade students 5. Violence in the workplace: videogames or access to weapons?

example of extraneous variable vs confound

As an example, if we manipulate the lighting in the room and examine the effects of that manipulation on workers' productivity, then the lighting conditions (bright lights vs. dim lights) would be considered the independent variable and the workers' productivity would be considered the dependent variable. If the bright lights are noisy then that noise would be a confound since the noise would be present whenever the lights are bright and the noise would be absent when the lights are dim. - If noise is varying systematically with light then we wouldn't know if a difference in worker productivity across the two lighting conditions is due to noise or light. So confounds are bad, they disrupt our ability to make causal conclusions about the nature of the relationship between variables. However, if there is noise in the room both when the lights are on and when the lights are off then noise is merely an extraneous variable (it is a variable other than the independent or dependent variable) and we don't worry much about extraneous variables. - This is because unless a variable varies systematically with the manipulated independent variable it cannot be a competing explanation for the results.

example of confound

Example of a confounding variable You collect data on sunburns and ice cream consumption. You find that higher ice cream consumption is associated with a higher probability of sunburn. Does that mean ice cream consumption causes sunburn? Example: Participant may think that the researcher is too young to be credible. Psychological characteristics if the researcher can affect the behavior of the participants. Example: Personality may be off putting or researcher may be in a bad mood.

TOPIC 2 INFO

RESEARCH VALIDITY

TOPIC 1 INFO

PSYCHOLOGY AND THE SCIENTIFIC METHOD

TOPIC 3 INFO

VARIABLES AND MEASUREMENT

random sample

a sample that fairly represents a population because each member has an equal chance of inclusion

reliability, like validity, is based on...

correlations

discriminant validity

different measures of different constructs should not be correlated or related to each other

convergent validity

different measures of the same construct should be correlated or related to each other

in class term - psychology behavioral science difference

different to hard science in the fact that human behavior is generally much more complex (i.e. greater variability) and that term-32measurement is often indirect - (e.g. there is nothing akin to a ruler that can be used to measure personality)

Ways of knowing (e.g., what are they, which ones are empirical, limitations of each)

from class notes step 1: observation (empiricism) step 2: develop a theory step 3: develop a testable, refutable hypothesis step 4: make systemic observations step 5: evaluate the evidence from internet 1. intuition 2. authority 3. rationalism - using logic and reasoning to acquire info 4. empiricism - info from observation and experience 5. scientific method - can require time and resources

coefficient alpha

inter-item consistency - this is a measure of inter-item consistency (i.e. the consistency of responses to all items on the test) - this is an indication of the extent to which each item on the test measures the same thing as every other item on the test - the more homogenous the domain (test), the higher the inter-item consistency

split half, odd-even or random split reliability

internal consistency - the primary issue here is one of obtaining comparable halves