Research Methods Study Guide 1
ordinal
a.higher scale of measurement than nominal b. different values of a variable can be ranked according to quantity c. mathematical operations are likely to produce misleading results d. adds some crude information e. the order matters but not the difference in values (nonparametric)
substantial overlap
between basic and applied research is evident
correlational internal validity
changes in value of criterion variable are solely related to the changes in value of the predictor variable
face validity
how a measurement instrument appears to measure what it is designed to measure
how do we acquire knowledge?
method of authority, method of tenacity, inductive logic, deductive logic, and rational method (using reason and logic)
correlational design
no independent variables are manipulated, two or more dependent variables are measured and a relationship established. it can be used for predictive purposes. it cannot be used to establish causal relationships among variables.
parallel-forms reliability
same as test-retest except the form of the test used on first administration is replaced during the second by a parallel form (ISSUE: if not equivalent, could cause differences in performance that reduces reliability)
further observing and experimenting (scientific method)
the point at which the scientific method differs from the other methods of inquiry
two ways to control extraneous variables
1. Hold them constant: if they do not vary, they cannot cause uncontrollable variation in dependent variables 2. Randomize their effects across treatment: deals with the effects that can't or shouldn't be held constant
which type of validity is more important?
1. If most interested in testing theoretical position (basic), then INTERNAL is more important. 2. If more concerned with applying results to a real world problem (applied), then EXTERNAL is more important
demand characteristics
1. cues provided by the researcher and research context gives participants information about a study 2. cues focused on by the participant may not be relevant to the study so it can adversely affect results
external validity
1. degree that results generalize beyond your sample and research setting 2. becomes more relevant when the findings are expected to be applied directly to world settings
scientific method
1. observing a phenomenon 2. formulating tentative explanations or statements of cause and effect 3. further observing or experimenting (or both) to rule out alternative explanations 4. refining and retesting the explanations
why is data collection a reactive process?
1. psychological study is a social situation. 2. participants' social history can affect how they respond to a study 3. simply observing someone can change their behavior (whether they realize it or not)
measurement scales
1. refers to units in which a variable is measured 2. can also be about some TYPE of scale represented by a given set of units 3. four basic scales which can be arranged in order of information provided about values. (type of scale along which a given variable is measured determine the kinds of statistical analyses that can be applied to the data)
internal validity
1. the ability of research design to adequately test your hypothesis 2. must be considered during the DESIGN phase of research 3. ability of your design to test the hypothesis it was designed to test
how to minimize reactivity
1. use a blind technique where the experimenter doesn't know which condition a participant has been assigned. 2. automate the experiment 3. use a double blind technique where neither experimenter nor participant knows the condition to which a participant has been assigned 4. include manipulation checks in the experiment
organizing and interpreting research results (3rd role of a theory)
a theory provides a framework for understanding research research results can be interpreted on a theory EVEN if the research was not specifically designed to test that particular theory
systematic observation
careful observation of real world behavior, published research reports, your own previous or ongoing research, peruse internet research. this research is planned. it's decided what you are going to observe, how you are going to observe it, and how you will record your observations. your own systematic observations of real world behavior can provide the basis for research idea PLANNED
method of authority
consulting some source that you consider to be authoritative on the issue in question (e.g. consulting books, television, religious leaders, scientists). The method plays an important role in the acquisition of scientific knowledge because the information obtained from authorities can familiarize someone with the problem, the available evidence, and the proposed explanations.
mundane realism
degree to which a simulation mirrors the real-world event (ex. courtroom realistically constructed in a lab)
experimental realism
degree to which simulations psychologically involves participants in an experiment simulation high in this may prove ability to generalize lab results in the real world (ex. Stanford Prison Project)
experimental approach
has the ability to identify and describe causal relationship. it can tell you whether changes in one variable actually caused changes in the other.
factors that affect internal validity
history, testing, maturation, instrumentation, biased subject selection, statistical regression, experimental mortality
content validity
how adequately the content of a test samples the knowledge, skills, or behaviors that the test intended to measure
Concurrent validity
if scores on test and criterion are collected at about the same time
concurrent vs. predictive
if scores on test and criterion are collected at about the same time VS. if scored on test and criterion are compared at a later time
unreliable
if the design is low in internal validity it is likely to be--
guiding the direction of research (4th role of a theory)
in addition to being organized by theory, research results can be interpreted in light of a theory. Theories are valuable because they often provide ideas for new research (known as the hueristic value of a theory)
randomization
in an experiment is important because is minimizes the possibility of bias responses. this ensures that each participant or subject has an equal chance of being placed in any group. it helps ensure that any differences between and within groups are not systematic at the end of the experiment
primary sources
includes a full report of a research study including methodological details (preferred) including all details necessary to duplicate the study. includes descriptions of the rationale of the study, its participants, materials, apparatus, procedure, results, and references. scholarly articles are the most important.
theory
is a partially verified statement of scientific relationship that cannot be directly observed. if the theory is stated formally, the statement consists of a set of interrelated propositions that attempt to specify the relationship between the variable and some behavior
independent variable
is manipulated (with at least two levels) manipulating it means exposing subject to at least two values or levels of a the variable ("treatments")
dependent variable
is measured. variable whose value you observe and measure in an experimental design. hope to show causal relationship between the values of the variables
directionality problem (correlational design)
it is not always possible to specify the direction in which a causal arrow points
rational method
logical reasoning rather than on authority or evidence of ones senses. Logically deduced conclusions from self-evident truths. If one (or both) of the assumptions used in the deduction process is incorrect, the logically deduced conclusion will be invalid. This method is not used to develop scientific explanations.
experimental internal validity
means showing that variation is ONLY the independent variable caused the observed variation in the dependent variable
randomization
minimizes potential biases ensures participants have an equal change of placement in the groups ensures that differences between and within groups are not systematic at the end helps to distribute the effects of extraneous variables evenly across the treatments and groups
reliability of judgments/ratings by multiple observes
need interrater reliability degree to which they agree (minimum of 80% acceptable)
retesting (step 4 of scientific method)
not all research studies produce affirmative results, sometimes the research does not confirm the hypothesis. in some cases researchers would discard the original hypothesis, in others they revise it and retest it. Revised and refined hypotheses must be tested as rigorously are the original.
unsystematic observations
observation of everyday behavior, observation of animal behavior, paying attention to research presented in your class, good way to discover a general research idea. one of the most potent sources of research ideas is curiosity about the causes or determinants of commonplace, everyday behavior NOT PLANNED
observing a phenomenon (step 1 of scientific method)
observe the behavior of interest through process of observation the researcher identifies variables that appear to have an important influence on behavior
parsimonious (characteristic of scientific method)
often more than one explanation is offered for an observed behavior. scientists prefer the parsimonious explanation or the one that explains behavior with the fewest number of assumptions.
scientific method vs. theory
one's steps are - observing a phenomenon, formulating tentative explanations (hypotheses), further observing and experimenting, and refining and retesting explanations VS. the other's roles are- understanding, prediction, organizing and interpreting research results, and generating research
books
ones that provide summaries would be considered secondary sources. they should be used with caution because they may not undergo rigorous review, and info may not be up to date. an article in an anthology may be an original work and thus can be treated as a primary source if it was reproduced exactly and not edited for the anthology.
reliability of population estimates
precision of estimate called the margin of error opinion, attitude, other psychological variables
conventions and professional meetings
provide the most up to date information. information is at the frontiers of science. expensive to attend
factors that affect external validity
reactive testing interactions between participant selection biases and independent variable reactive effects of experimental situations multiple treatment interference
secondary sources
summarized information from a primary source. (used sparingly because could be incomplete, biased, and/or inaccurate) includes review papers and theoretical articles that briefly describe studies and results as well as descriptions of research found in textbooks, popular magazines, newspaper articles, television programs, films, or lectures excellent starting point for the rest of research
observations vs. research
systematic and unsystematic are either planned or not and they observe real-world BEHAVIOR VS. applied research and basic research may or may not be interested in the APPLICATION to real-world PROBLEMS.
method of tenacity (2)
tenacious means persistent. information you heard but do not remember who it is from proves it is inaccurate.
internal validity vs. external validity
the ability of research design to adequately test a hypothesis VS. the degree that results generalize beyond your sample and research settings
reliability vs. validity
the ability to produce similar results when repeated measurements are made under identical conditions VS. extent to which the test measures what you intend it to measure
refining (step 4 of scientific method)
the process of generating new, more specific hypotheses in light of previous results illustrates the refinement process. confirming a hypothesis with the research study leads to other hypotheses that expand on the relationships discovered
formulating testable (tentative) explanations (step 2 of scientific method)
the tentative statement that is offered concerning the relationship between variables of interest is called hypothesis. Hypothesis links two variables by a statement indicating the expected relationship between them. Research hypotheses often take the form of a statement of how the changes in value of one variable will affect the value of the other variable
understanding phenomena for which they account (1st role of a theory)
theories represent a particular way to understand the phenomena with which they deal. To the degree that a theory models an underlying reality, this understanding can be deep and powerful
third-variable problem (correlational design)
there may be an unmeasured variable that actually causes changes in observed behavior. resolved by examining the effects of each potential third variable to determine it does actually account for the observed relationship.
components of experimental design
two defining characteristics: manipulation of one or more independent variables and control over extraneous variables (experimental and control)
split-half reliability
two parallel forms of the test intermingled on a single administration of one test
role of theory
understanding phenomena for which they account, providing basis for prediction, organizing and interpreting research results guiding the direction of research
extraneous variables
undesirable variables that influence the relationship between the variables that the experimenter is examining.
reliability of psychological tests or measure
variables tend to change naturally over time
testable (characteristic of scientific method)
verifiable through observation and can be disproved but NEVER proven. should be verifiable through direct observation or lead to specific predictions about what should occur under conditions not yet observed
construct validity
when a test is designed to measure a "construct" or variable, not directly observed, that has been developed to explain behavior or the basis of some theory (ex. intelligence, self-esteem, achievement, motivation)
correlational design is used
when gathering data in early stages of research, when manipulating a variable is impossible or unethical, when you are relating two or more naturally occurring variables can provide a rich source for hypothesis
when to use experimental design
when planning a study to meet specified objectives to check causality
why can't a theory be proven
It is a partially verified statement of a scientific relationship that cannot be directly observed. It can be supported with evidence but easily disproven when an experiment fails.
Basic research
a. conducted to evaluated (confirm or disconfirm) theories or empirical positions b. goal is to acquire information about phenomenon c. little emphasis on application to real-world problems d. ex: research on memory process may be conducted to test efficiency of interference as a viable theory of forgetting (interested in discovering something about forgetting process while testing validity of theoretical problem, less interested in real world application of forgetting)
rigorously evaluated
a. constantly evaluated for consistency with evidence, generality, and parsimony b. derived from the other characteristics listed. (characteristics of the scientific method)
method of authority (2)
a. continue to practice this until we die. b. someone is considered as a source of authority, usually an expert source c. examples of authority figures include parents, teachers, religious leaders d. while they are convenient, they are not infallible (they may not truly be authoritative, sources are often biased by a particular point of view)
rational method (2)
a. doctrine of rationalism comes from Descartes. b. valid conclusions about the universe are drawn through pure reason. c. Descartes searched for "self-evident" truths (MUST be true because to assume otherwise would contradict logic) (ex. "I think therefore I am") d. if assumptions are true and there are no logical errors made, deductions to empirical test
applied research
a. primary goal is to evaluate real world problems (distinction between basic and applied) b. hypothesis may come from a theory c. substantial overlap between basic and applied research is evident
interval
a. scale of measurement on which the spacing between variable is known b. no true zero (no absolute zero) c. CAN apply mathematical operations d. Cannot make ratio judgements (ex. Celsius scale for temperature) Parametric
control over extraneous variables
a. variables not of interest for present experiment but might affect behavior you wish to investigate. b. can produce uncontrolled changes in value of dependent variable (makes it difficult or impossible to detect any effects of independent variables, may produce chance differences in behavior across levels of the independent variable)
nominal
a. variables who differ in quality and not quantity are said to fall along this b. values have different names but no ordering of values is implied (ex. you can count the # of observations falling into categories but can't apply mathematical operations) c. yields least amount of information (nonparametric)
ratio
a.similar to interval scales b. can make ratio comparisons c. HAS a true zero point (absolute zero) d. can apply mathematical operations (ex. Kelvin scale for temperature) Parametric
characteristics of a good theory
ability to account for data, explanatory relevance, testability, prediction of novel events, parsimony
reliability
ability to produce similar results when repeated measurement are made under identical conditions. more variability = less reliability
parts of a manuscript
abstract, introduction, methods, results, discussion, and references
test-retest reliability
administering the same test twice separated by relatively long interval or time (issue: participants could respond the same way because they remember what they did or said the first time)
further observing and experimenting (step 3 of scientific method)
after formulating your hypothesis you design a research study to test the relationship you proposed
both ratio and interval scales
allow you to determine by how much the various data points differ
generalizable (characteristic of scientific method)
apply beyond the original observations on which they are based. scientist prefer explanations of broad explanatory power over those that "work" only within a limited set of circumstances
tentative (characteristic of scientific method)
are never accepted as absolutely correct... no absolutes. Scientists may have confidence in their explanations, but they are nevertheless willing to entertain the possibility that an explanation is faulty. Attitude is strengthened by even Newton's conceptions of the universe needing to be replaced with the discovery of new evidence
reliability of physical measure
assessed by repeatedly measuring a fixed quality of the variable and using observed variation to derive precision of a measure more precise = less range in variation
empirical (characteristic of the scientific method)
based on objective, systematic observations, often carried out under carefully controlled conditions. The observable events and conditions referred to in the explanation must be capable of verification by others. Based on evidence of the senses
methods of inquiry
before a scientist can offer valid and general explanations for behavior they must gather information about the behavior of interest. Knowledge about behavior can be acquired by: method of authority, method of tenacity, the rational method, and the scientific method.
predictive validity
by comparing the scores on your test with the value of a criterion measure observed at a later time. indicates the ability of a test to predict some future behavior. (ex. SATs predict future college performance)
criterion-related validity
How adequately a test score can be used to infer an individuals value on some "criterion" measure Compare values inferred from the test to criterion values actually observes
testability
a theory must be testable. must be capable of failing some type of empirical test
characteristics of the scientific method
empirical, rational, testable, parsimonious, generalizable, tentative, replicable
scientific explanations are
empirical- based on evidence of the senses, rational- follow the rules of logic and are consistent with known facts, testable- verifiable through observation and can be disproved but NEVER proven. parsimonious- provide the simplest explanation using the fewest possible assumptions generalizable- apply to beyond the original observations on which they are based tentative- never accepted or absolutely correct replicable- fluke or chance occurrence are not acceptable rigorously evaluated- constantly evaluated for consistency with evidence, generality, and parsimony
providing basis for prediction (2nd role of a theory)
even when theories do not provide a fundamental insight into the mechanisms of a behaving system (as descriptive theories do not) they at least provide a way to predict the behavior of the system under different values of its controlling variables. if it is a good theory the predictions will match the empirical outcome with a reasonable degree of precision
research designs
explanatory data collection and analysis (classify behavior, identifying important values, identifying relationships among variables) hypothesis testing (evaluating explanations for observed relationships, begins after enough information collected to form testable hypothesis)
validity
extent to which the test measures what you intend it to measure. important to social testing like IQ tests.
replicable (characteristic of scientific method)
fluke or chance occurrences are not acceptable
rational (characteristic of scientific method)
follow the rules of logic and are consistent with known facts. If the explanation makes assumptions that are known to be false, commits logical errors in drawing conclusions from its assumptions, or is inconsistent with established fact, then it does not qualify as scientific
