Combo "Educational Research-Chapters
Pearson r
(Product moment correlation coefficient) The most common and precise technique; appropriate when both variables are expressed as continuous (i.e. ratio or interval data)
Testing
(Protest sensitization) When taking a pretreat alters the results of a posttest (increased likelihood when time between tests is short)
Spearman rho
(Rank difference correlation) Used to calculate when at least one variable is expressed as rank or ordinal data (other data must be converted to rank); easier to compute when numbers are small than pearson r
Selection Methods
(Selection Items) Taker must select from among a set of given answers (mult choice, true/false, etc.)
Common Variance
(Shared variance) The extent to which variables vary in a systematic way; the variation in one variable that is attributable to its tendency to vary with another variable
Environmental Variable
(Such as learning material) A variable in the setting of a study that may cause unwarranted differences between groups
Participant Variable
(Such as reading readiness) A variable on which participants in different groups in a study may differ
Supply Methods
(Supply Items) Taker must supply an answer (short answer, fill in the blank)
Table of Random Numbers
(Table of random digits) A table utilized to select a random sample
Stability
(Test-retest reliability) The degree to which scores on the same test are consistent over time
Multiple Regression Equation
(multiple prediction equation) A prediction equation including two or more variables that individually predict a criterion Y=a+bX+bX+bX
What coefficients are adequate for individual prediction?
+ or - .8
What coefficients are adequate for group prediction?
+ or -.6 or .7
Difficulties concerning educational research
- Involves human beings - indirect measurement of the variables being studied - Difficulties generalizing from specific studies - Complications when observing
Characteristics of Quantitative Methods
- Numerical Data - Use of formally stated hypotheses and procedures - Large number of participating subjects - use of pencil and paper tests, questionnaires
Purpose of Measurement
- Obtain information about the variables being studied - Provide a standard format for recording observations, performances, or other responses of participants - Provide for a quantitative summary of the results from many participants
Probability
- Statistically driven sampling techniques where the probability of being selected is known - Purpose is to select a group of participants representative of the larger group of subjects from which they are selected (quantitative)
Equivalence and Stability (reliability evidence)
- Testing the same participants with two forms of the same test taken at different times - Limitation - difficulty in creating parallel forms
Stability (Reliability evidence)
- Testing the same subject using the same test on two occasions - Limitation - carryover effects from the first to second administration of the test i.e. test/ retest
Internal Consistency (Reliability Evidence)
- Testing the same subject with one test and "artificially" splitting the test into two halves to be given at two separate times. Think two test versions, A & B...is one harder than another? - Limitations - must have a minimum of ten (10) questions
Equivalence (reliability evidence)
- Testing the same subject with two parallel (i.e., equal) forms of the same test taken at the same time - Limitation - difficulty in creating parallel forms
Mean
- The arithmetic average of all scores - Statistical properties make it very useful - Concerns related to outlying scores
Standard Deviation
- The average distance of the scores from the mean - The relationship to the normal distribution
Reliability
- The extent to which scores are free from error...Test/Retest - Error is measured by consistency - Sources of Error: test construction and administration/ participant's characteristics
Agreement (Reliability Evidence)
- Used when traditional estimates such as stability, equivalence, equivalence and stability, or internal consistency are not applicable - Typically some form of agreement is used (e.g., raters agreeing with one another)
Assumptions of a Quantitative Researcher
- We live in a stable, uniform, and coherent world - We can measure, understand, and generalize about our world - Generally regarded as a positivistic perspective
Why should we learn about research?
-To become research literate -our society is driven by research -to improve critical thinking skills -to learn how to read and critially evaluate research -to learn how to design and construct research
What is the conventional cutoff point for statistical significance?
.05 (means 95% confidence)
Ethical Guidelines for Qualitative Researchers
1. Develop an ethical perspective that is close to your personal, ethical position; 2. Seek research participants' informed consent; 3. Determine the broader social principles that affect your ethical stance; 4. Consider confidentiality to avoid harm; 5. There is no room for deception!
Characteristics of Quantitative / Qualitative Research
1. Identify a research problem; 2. Review the literature; 3. Select participants/sample; 4. Collect Data; 5. Analyze / interpret data; 6. report and evaluate research
Characteristics of Scientific Inquiry
1. Objectivity in observation 2. Control of personal bias 3. Precision 4. Parsimony 5. Tentative conclusions 6. Verification 7. Openness to scrutiny 8. Logic
Shank's "visions" of critical thinking
1. the "mirror," where critical thinking is sharp and reflective 2. the "window," where critical thinking is simple and elegant 3. the "lantern," where critical thinking is flexible and creative
5 Categories of Research
1.) Basic 2.) Applied 3.) Evaluation 4.) R & D 5.) Action
5 Steps in the Scientific Method
1.) Recognition and defining of problem 2.) Formulation of hypotheses 3.) Collection of Data 4.) Analysis of Data 5.) Stating conclusions
4 Steps in the Educational Research Process
1.) Selection of a problem 2.) Use of specific research procedures to design and collect data 3.) analysis of data 4.) statement of conclusions
Franz Mesmer
1st psychology experiment, led by Ben Franklin in 1782: Placebo Control using "magnetized" water to heal... origin of the word "mesmerized"
Minimum Sample Size
30 participants (usually); 30 in each subgroup if analyzing multiple subgroups
Sample Size- Quantitative
30 participants for correlational research/ 15 participants in each group for experimental research
Probability
95% likelihood
Intuition
A belief without an empirical basis
Inductive reasoning
A bottom-up approach to reasoning, working from the specific to the more general.
Case
A bounded system
Cover Letter
A brief, neat, and (when possible) specifically addressed letter that explains what is being asked of the respondent and why while giving a good reason for cooperating, a commitment to share results, and place where the researcher can be reached
Universe
A broad category of cases or units to which the study findings apply
Assessment
A broad term that encompasses the entire process of collecting, synthesizing, and interpreting information, whether formal or informal, numerical or textual
Standard Score
A calculation that expresses how far a score is from the mean
Sampling Element
A case or unit of analysis of the population that can be selected for a sample
True Category
A category into which people or object naturally fall
Deductive Reasoning
A chain of logic progressing from general (e.g., theory) to specific (e.g., hypothesis)
Inductive Reasoning
A chain of logic progressing from specific (e.g., observations) to general (e.g., theory)
Organismic Variable
A characteristic of a subject or organism (e.g. age or gender); such a variable cannot be manipulated
b
A coefficient that indicates the contribution of the predictor variable to the criterion variable
Interpretive Research
A collective, generic name for qualitative approaches
Index
A composite measure that combines several indicators into a single score
Example of evaluation research
A computerized algebra system has been installed properly, is being used properly, and as a result, student achievement is increasing as a result of its use - this program is perceived to be an efficient use of funds
a
A constant calculated from the scores of all participants
Pearson r
A correlation coefficient used when both variables are expressed as interval or ratio data pg 316 r=[EXY-(EX)(EY)/N]/{sqroot[EXsquared-(EX)squared/N][EYsquared-(EY)squared/N]}
Subculture
A culture embedded within a larger culture.
ERIC
A database (clearinghouse) of journal and non-journal education literature and documents
Correlation Coefficient
A decimal ranging from -1.00 to +1.00 that illustrates the size and direction of the relation between variables
Statistical significance
A decision made from the results of inferential statistical procedures that enable researchers to conclude that the findings of a given study are large enough in the sample studied in order to represent a meaningful difference or relationship in the population from which the sample was drawn.
Operational Definition
A definition used to explain work being done (e.g. unstable and stable need to be defined more specifically in order to select comparison groups for a study)
Sampling Validity
A faction of content validity concerned with how well the test samples the total content area being tested
Item Validity
A faction of content validity concerned with whether the test items are relevant to the measurement of the intended content area
Reliability
A feature of measures- the method of measurements is dependable and consistent
Validity
A feature of measures; the concept of interest closely mates the method used to measure it.
Degrees of Freedom
A formula (differs by test; Pearson r is n-2) used to determine whether results are statistically significant
Multiple Regression Equation
A formula used when multiple independent variables are present to determine whether variables are related and the degree to which they are related (Step-wise)
Grounded Theory
A general methodology for developing theory that is grounded in data systematically gathered and analyzed.
Sample
A group of individuals, items, or events that represents the characteristics of the larger group from which the sample is drawn
Institutional Review Board
A group that evaluates a potential study for its ethical guidelines
Hidden population
A group that is very different to locate and may not want to found and therefore is different to sample
Critical Ethnography
A highly politicized form of ethnography written by a researcher to advocate against inequities and domination of particular groups that exist in society (including schools)
Research Hypothesis
A hypothesis predicting a difference or change
Null Hypothesis
A hypothesis predicting a lack of change
Population
A larger collection of units from which a sample is taken
Why is it important to have as large of a sample as possible?
A larger sample is likely to produce a better reflection of the mean and standard error.
Panel Survey
A longitudinal study In which the exact same participants are surveyed over time
Cohort Survey
A longitudinal survey in which the researcher studies a particular group at intervals over time (random sample 300 of 1400/random sample 300 of 1400 10 yrs later-Same population; different participants)
Follow-up Survey
A longitudinal survey that addresses development or change in a previously studied population
Effect Size
A means for identifying the practical strength of the conclusions about group differences or about the relationship among variables in a quantitative study. Effect sizes tell us how different the sample values are and allow us to make judgement as to whether this is significant based on our knowledge of measures, the participants, and the data collection.
Standard Error of Measurement
A measure of how often errors can be reasonably expected
Correlation
A measure of the relationship between two variables
Split-Half Reliabilty
A measure of whether the test remains consistent (divide in half and compare scores)
Scale
A measure that captures a concepts intensity, direction or level at the ordinal level of measurement
Fourfold Table
A method of displaying data to reveal a pattern between two variables, each with two categories of variation
Replicate and Extend
A model of research favoring the repetition of an earlier study with an additional feature that enhances its generalization
Conceptual Framework
A model that connects the important elements of the problem being investigated
Comparison Groups
A more accurate term in casual-comparative research for experimental and control groups (as nothing is actually being controlled)
Dummy Variable
A nominal variable transformed into an interval variable
Snowball Sampling
A non-random sample in which selection is based on connections in a pre-existing network
Purposive or judgmental Sample
A non-random sample in which you use many diverse means to select units that fit very specific characteristics
Correlation Coefficient
A number between -1.00 and +1.00 that measures the level to which to variables are related. (0.00=no relationship; -1.00=negatively correlated; +1.00=positively correlated
Variable
A placeholder that can assume any one of a range of values
Variable
A placeholder that can assume any one of a range of values; for example, intelligence, height, and test score
Sampling Distribution
A plot of many random samples, with a sample characteristic across the bottom and the number of samples indicated along the side
Target Population
A population specified in very concrete terms
Research Hypothesis
A predicted outcome based on theory or understanding often stated as: If A, then B
Analysis of Narrative
A process in which the researcher collects stories as data and analyzes common themes to produce a description that applies to all the stories captured in the narratives
Reflective teaching
A process of developing lessons with thoughtful consideration of educational theory, existing research, and practical experience, along with the analysis of the lesson's effect on student learning.
Interview
A purposeful interaction in which one person obtains information from another
Case Study Research
A qualitative approach in which researchers focus on a unit of study known as a bounded system
Case Study Research
A qualitative research approach to conducting research on a unit of study or bounded system (e.g. an individual teacher, a classroom, or a school can be a case)
Correlation
A quantitive measure of the degree of correspondence
Simpson's Paradox
A relationship found in groups combined will disappear or reverse itself when the groups are separated into sub-groups
Replication
A repetition of a study using different subjects to retest its hypothesis
Mixed-methods research design
A research design that combines both quantitative and qualitative research designs and data.
Researchable
A research hypothesis that can be tested directly by the collection and analysis of data
Relationship Study
A researcher attempts to gain insight into variables or factors that are related to a complex variable.
Experimenter Effects
A researcher's influences on participants or on study procedures
Random Sampling
A sample drawn in while a random process is used to select units from a population
Representative Sample
A sample that represents the intended population
Statistics
A set of procedures for describing, synthesizing, analyzing, and interpreting quantitative data
Stratum
A single strata, a subgroup within a sample (e.g. high school teachers or elementary teachers)
Constant
A single value of a variable (e.g. male is a constant of gender)
Interaction of History and Treatment Effects
A situation in which events extraneous to the study affect research results, also impacting generalizability
Sample
A small collection of units taken from a larger collection
Research design
A specific type of plan that will be used to carry out a research study.
Test Battery
A standardized test that measures achievement in several curriculum areas
Purpose of the Research Study
A statement of the researcher's intent or the objective of the study
Analysis of Covariance
A statistical method for equating randomly formed groups that do not perform equally well on pretest
Factor Analysis
A statistical procedure used to identify relations among variables in a correlation matrix (used to reduce a large number of responses or questions to a few more meaningful groupings)
Meta-Analysis
A statistical summary of studies conducted to answer the same research question
Norm-Referenced Scoring
A student's performance is compared to the performance of others (grading on the curve or percentile ranks)
Experimental Research
A study in which at least one independent variable is manipulated and other relevant variables are controlled
Review of literature
A summary of past and current research and thinking on a specific researchable topic
Cross-sectional Survey
A survey administered at a single point in time. A snapshot
Longitudinal Survey
A survey administered more than once with significant periods elapsing between each administration. A study over time
Census Survey
A survey in which researchers attempt to acquire information from every member of the population
Sample Survey
A survey that attempts to infer information about a population based on a representative sample
Measurement Scale
A system for organizing data so that it may be inspected, analyzed, and intepreted
Factorial Analysis of Variance
A technique allowing the researcher to determine the effects of the grouping or independent variable and the control variable both separately and in combination (allows analysis of all subgroups together rather than independently)
Matching
A technique for equating groups on one or more variables
Modernism
A term used by postmodernists to refer to anearlier and outdated period in the history of science that viewed the world as static a (unchanging) machine where everyone follows the same laws of behavior.
Tests of Significance
A test referencing a statistical level of probability at which we can confidently reject a null hypothesis
Logico-inductive analysis
A thought process used to analyze qualitative data that uses logic to make sense of patterns and trends in data.
Spearman Rho
A tool for measuring correlational data when examining ranked or ordinal data; produces a coefficient between +1.00 and -1.00
Instrument
A tool used to collect data
Deductive reasoning
A top-down approach to reasoning, working from the general to the more specific.
Trend Survey
A type of longitudinal survey that examines changes over time in a particular population defined by some trait or traits (Valedictorians surveyed each year)
Correlational Research
A type of non-intervention research that measures individual differences to uncover relationships between variables
Stratified Sampling
A type of random sampling in which a random sample is drawn from multiple sampling frames, each for a part of the population
Nonexperimental research
A type of research design where the researcher has no control over any variable in the study, either because it has already occurred or because it is not possible for it to be influenced.
Ex Post Facto
A type of research in which studies are done "after the fact" (not experimental)
Categorical Variable
A variable made up of different types of categories or phenomenon (gender)
Discrete Variables
A variable measured with limited number of fixed categories
Continuous Variables
A variable that can be measured with numbers that can be subdivided into smaller increments
Intervening Variable
A variable that cannot be directly observed or controlled can influence the link between predictor and criterion variables
Moderator Variable
A variable that changes the relationships among other variables
Quantitative Variable
A variable that differs in quantity or amount within a phenomenon (annual income)
Blind Scoring
A way for the researcher to minimize bias by scoring without knowing who is being scored
Stratified Sampling
A way to guarantee desired representation of relevant subgroups (strata) within the sample
Factor Analysis
A way to take a large number of variables and group them into a small number of clusters called factors by finding groups of variables that are correlated highly among each other but have weak correlations with other variables
Confidence interval
A zone, above and below the estimate from sample, within which a population parameter is likely to be
Types of Single-Subject Designs
A-B-A withdrawal (varied patterns); Changing Criterion (baseline followed by successive treatment phases with more stringent criterion for improvement); Multiple-Baseline (systematic addition of behaviors, subjects, or settings); alternating Treatments (rapid random change between treatments)
General Ethical Principles
A. Beneficence and Nonmaleficence; B. Fidelity and Responsibility; C. Integrity; D. Justice; E. Respect for people's rights and dignity
Family Educational Rights and Privacy Act of 1974
AKA the "Buckley Amendment"- was designed to protect the privacy of students' educational records
Simple Analysis of Variance
ANOVA (one way analysis of variance); Determines whether scores from two or more groups are significantly different at a probability level
F Ratio
ANOVA ratio with a numerator of variances between groups and a denominator of variances within groups
Skewed
Abnormal Distribution
What types of tests need to have high validities?
Achievement tests
Frequency Distributions
An organization of the data set indicating the number of times (i.e., frequency) each score was present. Types: Frequency table, Frequency polygon, Histogram
Thoery
An organized body of concepts, generalizations, and principles that can be investigated
ANCOVA
Analysis of covariance; used to deal with groups that vary on the pre-assessment (assumes participants have been randomly assigned to treatment groups)
Inferential Statistics
Analyze data from a sample to draw conclusions about an unknown population. We assess whether the differences of groups (their means) or the relationship among variables is much greater or less than what we would expect for the total population, if we could study the entire population.
data exhaust
Ancillary pieces of data that are digitized and surround a test score (time spent, skipped questions, ect)
Prospective Casual-Comparative Research
Another name for the alternative approach to casual-comparative research; begins with a cause and seeks its effects
Logistic Regression
Another type of multiple regression analysis used to examine the relationship between the predictor variables and dependent variable. The result is a prediction of whether the participant is a "case" or "non-case"
Population Parameter
Any characteristic of the entire population that you estimate from a sample
Relationship
Any connections between variables
Single-Variable Design
Any design involving one independent variable
Factorial Design
Any design that involves two or more independent variables
History
Any event occurring during a study that is not part of the experimental treatment but may affect the dependent variable
Cluster
Any location within which researchers find an intact group of population members with similar characteristics
Why doesn't a single study generally produce a broad generalization of results?
Any single study is limited in context and participants.
Hawthorne Effect
Any situation in which participants' behavior is affected by the knowledge they are participating in a study rather than the treatment
Action Research
Any systematic inquiry conducted by teachers, principals, school counselors, or other stakeholders in the teaching-learning environment that involves gathering information about the ways in which their particular schools operate, the teachers teach, and the students learn
Action Research
Any systematic inquiry conducted by teachers, principals, school counselors, or other stakeholders in the teaching-learning environment to gather information about the ways in which their particular schools operate, the teachers teach, and the students learn
Generalizability
Applicability of findings to settings and contexts different from the one in which they were obtained
Educational research
Application of the scientific method to educational topics, phenomena, or questions in search of answers.
Nonparametric Tests
Appropriate for ordinal or nominal scale data; when parametric assumption has been greatly violated; when nature of distribution is not known; less powerful
Deductive Reasoning
Arriving at specific conclusions based on general principles, observations, or experiences. General to specific
Besides the independent variable, what variables need to be the same among the (two) groups?
As many as possible (all)
What is the relationship between the limits and degree of confidence?
As the limits get farther apart, the degree of confidence increases.
Study of Values Instrument
Assesses the relative strength of a person's values in six different areas
Affective Test
Assessment designed to measure affective characteristics (emotion, attitude, etc.)
Casual Models
Assists the researcher to identify how things go together
What guides an experimental study?
At least one hypothesis that states an expected casual relation between two variables
Casual-comparative Research
Attempts to determine the cause or reason for existing differences in the behavior status of groups of individuals
Site-Ordered Predictor-Outcome Matrix
Attempts to order the case study sites in a manner that allows the researcher to understand the variables that appear to contribute most directly to the outcomes (e.g. factors that contributed to principals' decisions)
National Research Act of 1974
Authorized the creation of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, which was charged with developing an ethical code and guidelines for researchers
Mean
Average
Positivism/Post-positivism
Based on the assumption that phenomena should be studied objectively with the goal of obtaining a single true reality, or at least reality within known probabilities.
A Priori
Before the fact; comparisons planned for more than two groups before the research begins
Snookered
Being fooled while reading educational research
Placebo Effect
Beneficial effect caused by a person's expectations about a treatment rather than the treatment itself
Term for Multiple Modes
Bimodal, Multi-Modal
Structuralism
Broad or grand theory that emphasizes the importance of cultural, structural, institutional, and functional relations as providing a large part of the social world in which humans live and this structure is key in determining meaning and influencing human behavior.
Acceptable Level of Risk
Can be a determining factor in leaning towards type 1 or 2 errors; What will happen if I'm wrong?
Generalizability
Can it be applied to a larger population
Critical Thinking
Careful, astute reasoning that includes conceptualization, synthesis, analysis, evaluation, and the recognition of bias
Multiple-Treatment Interference
Carryover effects from an earlier treatment make it difficult to assess the effectiveness of a later treatment
The Hawthorne Effect or Novelty Effect
Change in behavior due to simply the awareness of being studied
Organismic Variable
Characteristic that cannot be altered but can be controlled for (sex)
Checklist Items
Check all that apply
Methods for Selecting Problem and Building Hypothesis
Choose a logical relation to investigate, suggested by theory or derived from experience; Do not conduct correlational treasure hunts to investigate too many variables.
Interval Variable
Classifies person or objects and ranks them in equal intervals (e.g. the difference between 30 and 40 is the same between 50 and 60)
Ordinal Variable
Classifies persons or objects and ranks them (Groups 1, 2, 3, 4, 5; Strongly Agree to Strongly Disagree)
t Test for Nonindependent Samples
Compare two groups formed by some type of matching
Random-digit dialing (RDD)
Computer based random sampling telephone numbers
Faulty Logic
Concluding that 2 events that occur at the same time are related
Applied Research
Conducted for the purpose of applying or testing a theory to determine its usefulness in solving practical problems
Basic Research
Conducted solely for the purpose of developing or refining a theory
Directional Hypothesis
Consider whether tests need to be one-tailed or two-tailed
Intraobserver Reliability
Consistency of single evaluator
Interobserver Reliability
Consistent among multiple evaluators
NonRandom Sampling Techniques
Convenience Sampling, Purposive Sampling, Quota Sampling
Artificial Dichotomies
Created by operationally defining a midpoint and categorizing subjects as falling above or below it (e.g. test scores of 50 or higher are high achievers, below are low achievers)
Other Terms for Dependent Variable
Criterion Effect or Posttest Variable
Types of Action Research
Critical and Practical
Constant Comparative Method
Data analysis in grounded theory research.
Inferential Statistics
Data analysis techniques for determining how likely it is that results obtained from a sample or samples are the same results that would have been obtained from the entire population
Missing Data
Data missing in the database because participants do not supply it.
Topic Statement
Defines variables of interest, specified relations among those variables, and, ideally, important characteristics of participants
Conceptual definition
Defining a variable or concept in theoretical terms with assumptions and references to other concepts
df
Degrees of freedom--larger samples have larger df; smaller samples have smaller df
Deductive Hypothesis
Derived from theory
Purpose of nonexperimental designs
Describe current existing characteristics such as achievement, attitudes, relationships, etc.
Do researchers make cases or describe data in the most meaningful way?
Describe data
Ethnographic Case Study
Describes the activities of a specific group and the shared patterns of behavior the group develops over time
Operational Definition
Describes the rule for "putting numbers next to names"
Operational Definition
Describes variables in measurable ways
Site-Ordered Descriptive Matrix
Descriptive data is included on a matrix, ordering the sites on the variable of interest (e.g. organizing based on how principals dealt with change)
Summative evaluation
Designed to make decisions regarding the overall quality of the program being evaluated
Prediction Studies
Designs that examine the predictive nature of the relationships between variables
Criterion-Related Validity
Determined by relating performance on a test to performance on a second test or other measure
Survey Research
Determines and reports the way things are; it involves collecting numerical data to test hypotheses or answer questions about the current status of the subject of study
Inductive Reasoning
Developing generalizations based on observation of a limited number of related events or experiences. Specifics to generalizations
Three Types of Replication
Direct (same investigator in specific setting) <simultaneous-same problem, location, and time>, systematic (different investigators, behaviors, or settings), clinical ( development of treatment package based on two or more interventions found to be effective)
Scientific Principle #6
Disclose research to encourage professional scrutiny and critique
Research Bias
Distortion of data collected in a research study or its non-objective inerpretation
Reliability
Does it measure what it measures (can it be repeated)
Internal Consistency Reliability
Does the test stay consistent with itself?
Return to Baseline
During action-research, a participants return to previous behavior when treatment is withdrawn
How do you calculate variance?
E(X-mean)squared/n
Mutually Exclusive Attributes
Each unit fits into one and only one category of a variable
Cognitive Measures
Educational tests that measure abilities and achievement
Affective Measures
Educational tests that measure non-cognitive constructs such as traits, attitudes, and beliefs
Reflective Practice
Educators thinking critically, collecting data to test ideas, and revising their practice
Ways to Distribute Surveys
Email, telephone, personal administration, and interview
Practical Action Research
Emphasizes more of a "how to" approach; less philosophical
Qualitative
Emphasizes natural settings, understanding, verbal narratives, and flexible designs. Usually takes place in naturally occurring situations.
Quantitative
Emphasizes numbers, measurements, deductive logic, control, and experiments. Behaviors and settings are controlled and manipulated.
Inference
Entails identifying relevant information and drawing appropriate conclusions
Ethics
Established guidelines that encourage responsible research practices and assure the protection research participants
Baseline Stability
Establishing a pattern or range of student performance
Formative evaluation
Evaluative decision making that occurs during the implementation of a project.
Summative evaluation
Evaluative decision making that occurs following the completion of a project.
Two-Tailed Test
Examines changes in both directions (e.g. Is there a difference between the behavior during the hour before lunch of kindergarten students who receive a midmorning snack and that of kindergarten students who did not receive a midmorning snack?--Could be better or worse)
Sampling Error
Expected variation among means taken from various samples due to the fact that no sample is exactly the same as its population
Positively Skewed
Extreme positive scores make mean higher mean>median>mode
Negatively Skewed
Extreme scores at lower end ( makes mean more negative) mean<median<mode
Variables
Factors that may affect the outcome of a study or characteristics that are central to the topic about which a researcher wishes to draw conclusions.
Treatment Diffusion
Treatment groups communicate with and learn from each other
conceptualization
Idea building
Other Terms for Independent Variable
Treatment, Casual, or Experimental Variable
Purposes of Relationship Studies
Identify related variables for subsequent examination in casual or experimental studies; Provide valuable information about variables for relationship studies (e.g. to remove variables)
Going Native
Identifying so completely with the group being studied that you can no longer remain objective.
Anonymity
Identities are kept completely secret from researchers
Free Response Items
In your own words
Case study
In-depth qualitative research study of an individual program, activity, person, or group.
Time-Ordered Meta-Matrix
Includes chronology as an organizing variable
Personality Inventory
Incorporates questions that describe behaviors characteristic of certain personality traits
Novelty Effect
Increased interest, motivation, or engagement simply because they are doing something different
Descriptive Statistics
Indicate general tendencies in the data (mean, mode, median), the spread of scores (variance, standard deviation, and range), or a comparison of how one score relates to all others (z scores, percentile rank).
Degrees of Freedom
Indicate to the researcher how much data was used to calculate a particular statistic; usually one less than the number of scores.
Variability
Indicates the spread of the scores in a distribution. Range, variance, and standard deviation all indicate the amount of variability in a distribution of scores.
Likelihood
Inferential statistics determine likelihood, not guarantees
Unstructured Interview
Informal conversation
Attitude Scale
Instrument that measure what an individual believes, perceives, or feels about self, others, activities, etc.
Cognitive Test
Intellectual processes such as thinking, memorizing, problem solving, analyzing, reasoning, and applying information are measured
Qualitative Sampling Techniques
Intensity Sampling, Homogenous Sampling, Criterion Sampling, Snowball Sampling, Random Purposive Sampling
Interjudge Reliability
Inter-rater reliability (two or more raters)
Intrinsic Case Study
Interest in understanding a specific case.
Instrumental Case Study
Interest in understanding something more general than the particular case.
Cognitive Illusion
Interpreting patterns among randomness
Intrajudge Reliability
Intra-rater reliability (one rater)
Intervening Variables
Intrude between he independent and dependent variables and cannot be directly observed but can be controlled for (anxiety, boredom)
Correlational Research
Involves collecting data to determine whether, and to what degree, a relation exists between two or more quantifiable variables
Correlational Research
Involves collecting data to determine whether, and to what degree, a relationship exists between two or more quantifiable variables
Review of Related Literature
Involves systematic identification, location, and analysis of documents containing information related to the research problem. The term is also used to describe the written component of a research plan or report that discusses the reviewed documents.
Evaluative Validity
Is the report unbiased and free from judgements
Problems With the Scientific Method
It cannot answer all problems, it cannot capture the richness of the environment under study, and all measuring instruments have a degree of error
Static-Group Comparison
Two no randomly formed groups receive different treatments and are posttested No Pretest is given to ensure similarity of groups
How many levels must the independent variable have?
Two or more
Ethnographic study
Type of qualitative study that attempts to describe social interactions between people in group settings.
Phenomenological study
Type of qualitative study that engages the researcher in a long process of data collection in order to fully understand a phenomenon.
Grounded theory
Type of qualitative study, attempting to discover a theory that relates to a particular environment.
Experimental research
Type of research design where the researcher has control over one or more of the variables included in a study that may somehow influence the participants' behaviors.
Data Analysis Method of Single-Subject
Typically examining visual
Instrumentation
Unreliability or lack of consistency in measuring instruments that results in invalid assessment of performance (e.g. Change I difficulty)
Scientific Principle #3
Use Methods that permit direct investigation of the question
Data Mining
Use analytical tools to identify and predict patterns in datasets or large warehouses of data that have been collected from thousands of subjects and hundreds of variables
Triangulation
Use multiple methods to obtain a more complete picture
Causal comparative studies
Use of correlational models to investigate possible cause and effect relationships
Cherry Picking
Use of selected research findings that support one's favored conclusion
Positivism
Used by qualitative researchers to refer to what might be better labeled "scientism," which is the belief that all true knowledge must be based on science.
Chi Square
Used to compare group frequencies--that is to see if an event occurs more frequently in one group than another
Path Analysis
Used to determine the degree to which predictor variables interact with each other and contribute to the variance of the dependent variables
t Test
Used to determine the statistical significance of one mean difference (2 means) first used to test beer recipes
T Test
Used to determine whether the scores of two groups are significantly different from one another
t Test
Used to determine whether two groups of scores are significantly different at a selected probability level (compare actual difference between means with expected difference by chance) Is this significant, or is this result due to chance?
Aptitude Test
Used to predict how well an individual is likely to perform in a future situation
Site-Ordered Effects Matrix
Used to sort through sites and display probably cause and effect relations among the focus of the study and its outcomes
Analysis of Variance
Used to test for significant differences among the scores for three or more groups
Phi Coefficient
Used when both variables are expressed in terms of a dichotomy such as gender, political affiliation, smoking status, etc. (categories are labeled 1 and 0 or 1 and 2, though the numbers have no numerical value)
Eta Coefficient
Utilized with curvilinear relation
Restriction of a Dependent Variable
It must represent a measurable outcome
Evaluation
Judgement
Ethnocentrism
Judging people from a different culture according to the standards of your own culture.
Sources of Weakness in Comparative Studies
Lack of randomization, manipulation, and control
Quantitative Research
Largely deductive research using numerical values rather than explaining complex phenomena through verbal descriptions
Qualitative Research
Largely inductive research aimed at explaining complex phenomena through verbal descriptions rather than data
What is the relationship between statistical significance and sample size?
Larger sample, lower statistical significance; smaller sample, higher statistical significance
What can diminish the accuracy of prediction?
Length of time between predictor and criterion measures
Scaled Items
Likert and semantic differential (on a scale of one to ten)
Personal Experience
Limited to personal experience, Much of our knowledge depends on what we have observed and how we have interpreted it. Bias. Natural inclination to protect self esteem.
Scientific Principle #2
Link Research to relevant Theory
Null Hypothesis
Make predictions that there will be no statistically significant difference between the independent variable and the dependent variable.
Experimental research
Manipulation of factors that may influence subjects.
Techniques for Controlling Extraneous Variables When Randomization Is Not an Option
Matching, Comparing Homogeneous Groups or Subgroups, Participants Serving As Their Own Controls, Analysis of Covariance
Three Control Techniques for Comparative Studies
Matching, Comparing Homogenous Groups or Subgroups, and Analysis of Covariance
Restricted Range of Scores
May lead to underestimation of true relation between variables (e.g. Measure achievement related to IQ in college students--most college students have higher IQs)
Which is generally the more preferred measure of central tendency?
Mean
Measures of Central Tendency
Mean, Median, Mode
Interpretive Validity
Meaning attributed to behaviors or words of participants
Ratio Variable
Measurement scale has a true zero point (height, weight, etc.)
Achievement Test
Measures an individuals current proficiency in given areas of knowledge or skill
Self-Referenced Scoring
Measuring how a student's performance changes over time
Affective Characteristic
Mental characteristic related to emotion, such as attitude
Cognitive Characteristic
Mental characteristic related to intellect, such as achievement
F test
Method of comparing complex groupings (based on diff. methods) ex. soil vs. hydroponics in crops
Frequently Manipulated Variables
Method of instruction, type of reinforcement, arrangement of learning environment, type of learning materials, and length of treatment
Probability Sampling
Method of sampling in which participants are selected randomly from a population in such a way that the researcher knows the probability of selecting each participant.
Non- Probability Sampling
Method of sampling in which the probability of selecting a participant is unknown -Method of sampling in which the probability of selecting a participant is unknown
Median
Middle Score (average middle numbers if even number of scores)
Relation for Coefficient between +.35 and +.65 or between -.35 and -.65
Moderate
Why would a researcher use a single-subject model?
Moral objections (withholding treatment, for example)
Parametric Test
More powerful, generally preferable, requires the following: 1) Measured variable must be normally distributed in the population 2) Data represent an interval or ratio scale of measurement (Some ordinal data like Likert scale may be included) 3) Selection of participants in independent (selection of one subject does not affect selection of another) 4) Variances of comparison groups are equal (or at least ratio of variances is known)
Problems With Matching
Mortality (be less precise)
z Score
Most basic and used standard score-how many standard deviations a score is from the mean z=(X-mean)/SD where X is the raw score
Mode
Most frequent number
Baseline Measures
Multiple measures of Pretest performance
Degrees of Freedom for Pearson r
N-2 (number of participants)
Types of Qualitative Research
Narrative Research, Ethnographic Research, Case Study Research
Sampling Error
Naturally occurring problem in which some parts of the population are underrepresented due to random sampling
Semantic Differential
Necessary __ __ X __ __ Unnecessary
Is it necessary to select a large number of samples from a population to estimate the standard error?
No
Non-experimental research
No manipulation of factors that may influence subjects. Usually means that the study can only describe something or uncover relationships between two or more factors.
Should the narrative researcher become an informant?
No!
Do correlational studies have to focus solely on one variable?
No, correlational studies typically investigate a number of variables believed to be related to a major, complex variable.
Can correlational studies indicate casual relationships?
No, they can lead to casual-comparative studies or experimental studies by sorting variables, but they cannot designate cause.
Quasi-Experimental Design
Non equivalent Control Group Design (2 groups pre and post; intact groups instead of random); Time-Series Design (repeated Pretests and posttests with control group or not); counterbalanced Design (groups receive treatment in different order);
Casual Comparative Research
Non-intervention research aimed at uncovering relationships by comparing groups of people who already differ on a variable of interest
Quota Sampling
Non-random sample in which you use any means to fill preset categories that are characteristics of the population
Chi Square
Nonparametric test of significance appropriate when data are in the form of frequencies or percentages and proportions that can be converted to frequencies
Nonprobability Sampling
Nonrandom sampling; Does not allow the researcher to specify the probability that each member of the population has of being selected
Freedom from Harm
Not exposing participants to undue risks
Does it matter if the t Test score is negative?
Not really; SPSS subtracts the second mean from the first automatically
Field Notes
Notes about what is viewed/experienced
Participant Observation
Observer is a part of what is being observed (Active Participant Observer, Privileged Active Observer, Passive Observer)
Nonparticipant Observation
Observer is outside of what is being observed
Privileged, Active Observer
Observing a setting at a time when one is not participating in an instructional setting (e.g. a teacher's aide observing a lesson)
ecological fallacies
Occur when group data are used to make an inference about an individual
Theoretical Saturation
Occurs when no new information or concepts are emerging from the data and the grounded theory has been validated.
Type I Error
Occurs when the null hypothesis is rejected by the researcher when it is actually true. The probability of this error rate is called "alpha."
Type II Error
Occurs when the research fails to reject the null hypothesis when an effect actually occurs in the population. The probability of this error is called "beta."
Quartile Deviation
One half of the difference between the upper quartile and the lower quartile in a distribution (subtract cutoff points)
Standardized Test
One that is administered, scored, and interpreted in the same way no matter when or where it is used
Pre-experimental Design Types
One-Shot Case Study, One-Group Pretest-Posttest Design, Static-Group Comparison
Single-Variable Rule
Only one variable should be manipulated at a time
Artificial Category
Operationally defined by a researcher (e.g. tall vs short)
Two Participant Variables
Organismic and Intervening
Etic Terms
Outsider's words or special words that are used by social scientists.
t Test for Independent Samples
Parametric; Does a difference exist between two independent samples (samples randomly formed without any kind of matching)
Two Variable Types That Must Be Controlled
Participant and Environmental
Pretest-Treatment Interaction
Participants respond differently to a treatment because they have been pretested
Ceiling Effect
Participants who score well on pre-tests have little room to improve
Personal-Attributes Effects
Passive experimenter effects caused by personality
Simple Predictive Studies
Performance on one variable (i.e., the predictor) is used to predict performance on a second variable (i.e., the outcome or criterion)
Participant
Person from whom data are collected - Can be thought of as source of evidence or unit of study - Also referred to as a subject
5 Ways we can know something
Personal Experience, Tradition, Experts, Inductive reasoning, Deductive reasoning, Scientific Method
Maturation
Physical, intellectual, or emotional changes that occur naturally within individuals over a period of time
Data
Pieces of information you collect and use to examine your topic, hypothesis, or observations
Raw Score
Point value of items a person answered correctly
Scientific Principle #1
Pose Significant Questions that can be investigated
Interaction of Time of Measurement and Treatment Effect
Posttest may yield different results depending on when it is done
Non-probability
Pragmatically driven sampling techniques where the probability of being selected is not known (quantitative)
Three Classifications of Single-Variable Designs
Pre-experimental, true experimental, and quasi-experimental
Reconnaissance
Preliminary Information Gathering
Level of Significance
Preselected probability level; represented by alpha (a)--usually 5/100 (.05)
True Experimental Design
Pretest-Posttest Control Group Design (ROXO); Posttest-Only Control Group Design (RXO); Solomon Four-Group Design (two pretested, two not)
p
Probability of chance (lower means less likely to occur due to chance)
Data Mining
Procedures for generating useful information from raw data in educational settings
Teacher empowerment
Process highlighted by teachers collecting their own data in order to assist in making decisions about their own students and classrooms; concept also known as "teacher as decision maker."
Triangulation
Process of relating multiple sources of data in order to establish their trustworthiness; used with qualitative research methods.
Canonical Analysis
Produces a correlation based on a group of predictor variables and a group of criterion variables
Personal Administration Pros and Cons
Pros: Efficient when respondents are closely situated Cons: time consuming, administrators must be trained
Online Survey Tools Pros and Cons
Pros: Many free tools exist, may provide analysis of items, will standardize format; Cons: not all tools are well-developed (garbage in, garbage out), not all are free and easy
Interview Pros and Cons
Pros: can probe, follow up, and explain questions, usually high return rate, may be recorder for transcription and analysis, flexibility of use; Cons: time-consuming, no anonymity, possible interviewer bias, complex scoring of unstructured items, administrators must be trained
Telephone Pros and Cons
Pros: high response rates, quick data collection, can reach a range of locations and respondents; Cons: requires phone number lists, difficult to get in-depth data, administrators must be trained
Mail Pros and Cons
Pros: inexpensive, confidential or anonymous, easy to score most items, standardized items and procedures; Cons: small response rate, cannot probe items, limited respondents who can read, possibility of response sets
E-Mail Pros and Cons
Pros: speedy results, easy to target responses, other advantages same as mail; Cons: not everyone has e-mail, possibility of multiple replies from single participant, other disadvantages same as mail
Scientific Principle #4
Provide a coherent and explicit chain of reasoning
Descriptive Statistics
Provide basic information about the number of participants, their characteristics, and how hey did on a test or outcome
Analysis
Pulling apart
Synthesis
Putting together
Are qualitative or quantitative methods more appropriately applied to action research?
Qualitative
Concurrent Sampling
Quantitative and qualitative portions are done independently
Time Series Quasi-experiments
Quasi-experiment in which a group is observed (or some measure collected) for a determined time, followed by intervention, and then observation or measurement is continued
Matching
Random assignment of matched pairs to subgroups (one to each)
Purposeful Random Sampling
Random sampling first, then the cases become purposeful
What is the best way to control for many extraneous variables simultaneously?
Randomization
Semantic Differential
Rank on a scale of two opposite terms: Strong ___ _x_ ___ ___ ___ Weak
Rating Scale
Rate choices, select which one is best
Interpretive/Constructivist
Reality is socially constructed; multiple realities, each related to the complexity of naturally occurring behavior, characterized by the perspectives of the participants. Also known as qualitative.
Deductive Reasoning
Reasoning from the general to the specific, limitations are that you only organize what you already know
Inductive Reasoning
Reasoning from the specific to the general, limitations are that in order to be certain, you must have observed all examples and datea
Attenuation
Reduction in correlation coefficients that tends to occur if the measures have low reliability
Rigor
Refers to the quality, validity, accuracy, and credibility of action research and its findings.
Conceptualization
Refining an idea by giving it a very clear, explicit definition
Statistically Significant
Relationships that are not likely to be due to chance
Mixed-Method studies
Relies on both and hybrid approaches of quantitative and qualitative procedures (sampling procedure)
Scientific Principle #5
Replicate and generalize across studies
What is the key to generalizability of a single-subject study?
Replication (the more replications, the more generalization)
Thurstone Scale
Requires participants to select from a list of statements that represent different points of view
Interest Inventory
Requires people to indicate personal likes and dislikes
Guttman Scale
Requires respondents to agree or disagree with a number of statements; it is then used to determine whether an attitude is unidimensional
Structured Item
Requires respondents to choose among provided response options (Scaled, Ranked, Checklins)
Descriptive Research
Research aimed at describing the characteristics of a population without generalizing or testing statistical hypotheses
Small-scale Evaluation Research
Research aimed at evaluating local programs or procedures for the purpose of improvement or decision making
Inferential Research
Research aimed at generalizing to a larger population with data collected from samples of the population
Single-subject Research
Research aimed at studying a single individual (or very small group) to learn more about relationships between variables
Group Research
Research aimed at studying one or more large groups to learn more about relationships among variables
Theory-based Research
Research aimed at testing hypotheses spun from a theory with the intention of evaluating the theory
Action (teacher) Research or Reflective Practice
Research conducted by practitioners in applied fields for the purpose of solving a specific problem
Meaningful Research
Research conducted with clear value assessed in terms of positive impact
True Experimental Research
Research coupling a manipulated independent variable (intervention) and random assignment to groups
Experimental Research
Research in which a researcher manipulates at least one independent variable, controls other relevant variables, and observes the effect on one or more dependent variables
Experimental Research
Research involving an intervention or treatment (manipulation)
Non-intervention Research
Research involving the search for relationships without changing conditions
Quasi-experimental Research
Research involving the use of a manipulated independent variable (an intervention) without random assignment or participants to groups
Discrepant Findings
Research outcomes that are inconsistent (or in opposition) across studies
Informed Consent
Research participants enter the research of their free will and with understanding of the nature of the study and any possible dangers that may arise as a result of participation
Qualitative Research
Research relying primarily on the collection of qualiative data.
Quantitative Research
Research that gathers numeric data through controlled procedures and analyses to answer predetermined questions or test hypotheses.
Case Study Research
Research that provides a detailed account and analysis of one or more cases.
Qualitative Research
Research that seeks to probe deeply into the research setting to obtain in-depth understandings about the way things are, why they are that way, and how the participants in the context perceive them
Large-scale Policy Research
Research using large data sets with the intention of shaping public policy and reform
Hypothesis
Researcher's predictions of the research's findings
Observational Measures
Researcher-recorded data based on personal perceptions of research participants or objects
Reject, Accept, Support and Prove Hypothesis
Researchers either reject a hypothesis or say it was supported; they never prove or accept a hypothesis
Confidentiality
Researchers know participants identities but do not divulge them in any way
Unstructured Item
Respondent has complete freedom of response (Free Response)
Projective Test
Respondents project their true feelings or thoughts onto an ambiguous stimulus
exception fallacies
Result from generalized conclusions based on a single case
Standard Deviation of a Sample
SD (sample-based statistics)
Standard Error of the Mean Formula
SEX=SD/SQ Root (N-1) SD=Standard Deviation for a Sample N=Sample Size
Multilevel
Sample is representative of different levels, e.g., classrooms, schools, and districts.
Systematic Sampling
Sampling in which every Kth individual is selected from a list (only random when list is randomly ordered)
Basic Procedure for Correlational Research
Scores for two (or more) variables of interest are obtained for each member of the sample, and the paired scores are then correlated.
Norm-Referenced Scoring
Scoring by comparing to other scores
Cross-Case Analysis
Searching for similarities and differences across multiple cases.
Snowball Sampling
Selecting a few people who fit a researcher's needs, then using those participants to identify additional participants, and so on
Criterion Sampling
Selecting all cases that meet some set of criteria or have the same characteristic (e.g. students who have been held back for two years)
Multistage Sampling
Selecting clusters within clusters
Cluster Sampling
Selecting intact groups, not individuals, randomly for a study
Random Purposive Sampling
Selecting more participants than necessary; then, randomly selecting participants from that list
Homogenous Sampling
Selecting participants who are very similar in experience, perspective, or outlook
Intensity Sampling
Selecting participants who permit study of different levels of the research topic (high/low)
Pilot Testing
Send a few items into a small number of people to test them
Standard Deviation of a Population
Sigma (o) (population parameter)
Alpha Level
Significance level. Probability level that reflects the maximum risk you are willing to take that any observed differences are due to chance.
Discriminant Function Analysis
Similar to multiple regression analysis but continuous predictor variables are used to predict a categorical variable
One-Shot Case Study
Single group is exposed to treatment (X) and then post tested (O) (no Pretest-little control)
One-Group Pretest-Posttest Design
Single group is pretested (O), exposed to a treatment (X), and tested again (O) History, maturation, and statistical regression are not controlled
Sample Size Guidelines
Small Populations (100 or fewer)=Entire Population; Medium Populations (500, give or take)=50%; Larger Population (1,500)=20%; Very Large Populations (5,000)=400 members is sufficient
One-Tailed Test
Solely examines outcomes in one direction (e.g. Does having breakfast improve test scores?--only seeks improvement)
Authority
Someone who has unique experience or expertise in something and is able to provide insights and understandings that we are unable to see. Prone to disagreements; authorities can be wrong.
Tradition
Source of answers to questions; based on past behaviors.
Emic Terms
Special words or terms used by the people in a group.
Standard Deviation
Square root of variance
Formula to Determine Common Variance
Square the correlation coefficient (e.g. .80*.80=.64=64% common variance)
Alternative Casual-Comparative Approach
Start with a cause and seek its effects (What is the effect of X?)
Basic Casual-Comparative Approach
Start with an effect and seek its possible causes
Nondirectional Hypothesis
States that a difference will be present
Directional Hypothesis
States that a specific difference will be present
Null Hypothesis
States things will not change
Conceptual Hypothesis
Stating a hypothesis with variables as abstract concepts
Meta-Analysis
Statistical approach to summarizing the results of many quantitative studies that have investigated basically the same thing
Inferential Statistics
Statistical reasoning that permits generalization beyond the sample to a larger population
Descriptive statistics
Statistical techniques that allow researchers to summarize, organize, and simplify data.
Inferential statistics
Statistical techniques that allow researchers to test the statistical significance of the difference between two or more groups or to test the degree of relationship between two variables.
Stratified Purposeful Sampling
Stratified sampling first, then a few cases examined intensely. Purposeful sampling first, then stratify on a variable
Relation for Coefficient between +.65 and 1.00 or between -1.00 and -.65
Strong
Likert Scale
Strongly Agree to Strongly Disagree
Likert Scale
Strongly agree to strongly disagree
Interview Types
Structured and Unstructured
Descriptive Studies
Studies that describe a phenomena. Criteria for evaluating descriptive studies: -Conclusions about relationships should not be drawn -Participants and instruments should be described completely
Multiple Regression
Studies that examine performance on several variables (i.e., predictor variables) to predict performance on a single outcome variable (i.e., criterion). Sample size of at least 10 subjects for each predictor variable. Relationships among the predictor variables (i.e., colinearity)
Simple Correlation Studies
Studies that examine the relationship between two variables. Two variables : -Predictor and criterion -Use caution describing the variables as independent and dependent
Ex-post-facto studies
Studies that investigate the relationships between independent and dependent variables in situations where it is impossible or unethical to manipulate the independent variable
Selection-Treatment Interaction
Study findings apply only to the groups involved in a study and not to the population as a whole; mainly occurs when participants are not randomly selected
Collective Case Study
Studying multiple cases in one research study.
Sigma (E)
Sum it up
Central Tendency
Summary numbers that represent a single value in a distribution of scores. They are expressed as an average score (the mean), the middle of a set of scores (the median), or the most frequently occurring score (the mode).
Types of Quantitative Research
Survey research, correlational research, casual-comparative research, and experimental research
SQ3R
Survey, Question, Read, Recite, Review
Culture
System of shared beliefs, values, practices, etc; that a group uses to understand their world and relate to others.
Educational Research
Systematic inquiry applied educational problems.
Research
Systematic process of gathering and analyzing information. Information is gathered on a topic directly from individuals, groups, documents, and other sources.
Sampling Bias
Systematic sampling error caused by the researcher
Scientific method
Systematic, step-by-step method for investigating questions and resolving problems.
Probability Sampling
Techniques that permit the researcher to specify the probability or chance that each member of a defined population will be selected for the sample (Simple random sampling, stratified sampling, cluster sampling, and systematic sampling)
Hypothesis
Tentative but intelligent and informed guess about the findings of a study, made before the study begins.
Grouping Variable
The "independent" variable being studied--in a casual-comparative study, these either cannot be manipulated (e.g. gender), should not be manipulated (e.g. number of cigarettes smoked a day), or are not manipulated (e.g. method of reading instruction)
Power
The ability of a significance test to identify a true research finding
Reflection
The act of critically exploring what you are doing, why you decided to do it, and what its effects have been.
Variance
The amount of spread between scores
Generalizability
The applicability of research findings to settings and contexts different from the one in which they were obtained.
Educational Research
The application of the scientific method to study educational problems - the goal is to explain and predict educational phenomena
Grouping Variable
The behavior or characteristic believed to influence some other behavior or characteristic
Dependent Variable
The behavior or characteristic that changes as a result of another variable
Research Topic
The broad area a researcher wishes to investigate
Heuristic
The case study illuminates the readers understanding of the phenomenon under study beyond the reader's original knowledge (gives new insight)
Particularistic
The case study is focused on a particular phenomenon, such as a situation or an event
Grouping Variable
The cause from causal-comparative research is a behavior or characteristic believed to influence some other behavior or characteristic
Dependent Variable
The change or difference in a behavior or characteristic that occurs as a result of the independent or grouping variable
Correlation Maneuver
The closer to 1.00 a correlation coefficient is, the stronger the link or association between variables being correlated
Quantitative Research
The collection and analysis of numerical data to describe, explain, predict, or control phenomena of interest
Ethnology
The comparative study of cultural groups.
Research Problem
The context for research that makes clear why data are being collected and often expressed as a gap in the knowledge
Shared Values
The culturally defined standards about what is good or bad or desirable or undesirable.
Control
The idea that procedures used in research can minimize bias, neutralize threats to validity, and rule out alternative explanations
Holism
The idea that the whole is greater than the sum of its parts.
Malleable Variable
The independent variable (one we can change)
Sample
The individuals selected from a population for study
Emic Perspective
The insider's perspective.
Population
The larger group from which the sample is taken
What differentiates experimental research from other types of research?
The manipulation of an independent variable
Confirmation Bias
Valuing support for a claim at the expense of disproof
Dependent Variable
Variable that depends on or is caused by another variable (criterion variable, effect, outcome, posttest)
Independent Variable
Variable that manipulates the dependent variable (experimental variable, manipulated variable, cause, treatment variable)
Factorial Design
Variation of single-variable design that permits investigation of two or more variables to determine if effects of independent variable are generalizable across all levels or are specific to particular levels
What is the relationship between control and external validity?
The more controlled the study, the less generalizable it will be; however, a study that is not internally valid is worthless
Interview
Verbal survey questions
What is the likelihood that any sample is exactly identical to its population?
Virtually nil
Scatterplot
Visual display of data
Tradition
Way of knowing where you do things as they hav always been done, limited based on your past
Personal Experience
Way of knowing where you rely on one's knowledge of prior experience - Limited by situations
Experts
Way of knowing where you rely on the expertise or authority of others, limitations are that they can be wrong, second opinions, etc...
Unobtrusive Measures
Ways to collect data that do not intrude on or require interaction with research participants
Tradition
We accept what has always been done as the best or right way. Difficult to change.
Retrospective Casual-Comparative Research
The most basic form of casual-comparative research; a researcher begins with an effect and investigates causes
Control Variable
The non manipulated variable in a factorial design
Frequency
The number of times something occurs
Reliability Coefficient
The numeric expression of reliability
Coefficient of Stability
The numeric expression of stability
Percentile Ranks
The percentage of scores that fall at or below a certain scorea
Target Population
The population the researcher wants to study
Y
The predicted criterion score for an individual
Statistical Significance
The probability that the results would have occurred simply due to chance
Confidence Limits
The probable limits within which the population mean falls (Estimated using SEX, Sample Mean, and normal curve)
Restorying
The process in which the researcher gathers stories, analyzes them for key elements of the story, and rewrites the story to place it in a chronological sequence
Comparing Homogenous Groups or Subgroups
The process of comparing two groups or subgroups that are similar; this process can limit generalizability (comparing subgroups--e.g. High IQ, Mid IQ, Low IQ--may be preferable)
Source of Knowledge: Authority
We obtain knowledge because a trusted authority told us
Relation for Coefficient between +.35 and -.35
Weak
Volvo Effect
Wealth linked to higher achievement
Example of a Casual-Comparitive Quantitative Design
What is the affect of attending a a magnet school on student attitude? What characteristics differentiate students who drop out of those who do not?
example of single subject quantitative design
What is the effect of a behavioral training program on Joan's ability to complete her performance tasks?
Example of Experimental Quantitative Design
What is the effect of teaching with manipulative vs. a traditional algorithm approach on students test scores
Examples of a Correlational Quantitative Design
What is the relationship between ACT Scores and Freshman grades?
Theoretical sensitivity
When a researcher is effective at thinking about what kinds of data need to be collected and what aspects of the collected data are most important for the grounded theory.
Interact
When a variable affects a dependent variable
When should a researcher use the median?
When extreme scores would skew mean results
Data Saturation
When the researcher begins to hear the same thoughts, perspectives, and responses from most or all participants, additional participants are not needed
Single-Subject Experimental Design
When the sample size is one or when individuals are treated as one group
Prediction Study
When two variables are highly related, scores on one variable can be used to predict scores on the other variable. An attempt to determine which of a number of variable are most highly related to the criterion variable
Measures of Relative Position
Where a score falls in a distribution relative to all other scores
Questionnaire
Written questions for a survey
Realist Ethnography
Written with an objective style and uses common categories for cultural description, analysis, and interpretations (e.g. family life, work life, etc.)
Prediction Equation (Single Variable)
Y=a+bX
Diagnostic Test
Yields multiple scores to facilitate identification of a student's weak and strong areas
T Score
Z score; a z score expressed as an equivalent score T=z(10)+50
Measurement
The process of quantifying or scoring performance on an assessment instrument
Research and Development
The process of researching consumer needs and then developing products to fulfill those needs (not to formulate or test theory but to develop effective products)
Pairwise Matching
The process of selecting a participant from the second comparative group that matches a participant from the first in terms of control variable(s) (Those with no pairs are eliminated from the study); this process can lead to small numbers, particularly when controlling for several variables
Sampling
The process of selecting a sample
Quota Sampling
The process of selecting a sample based on required, exact numbers, or quotas [5 math teachers, 3 science, 5 English]
Simple Random Samplng
The process of selecting a sample in such a way that all individuals have an equal and independent chance of selection for the sample (highest chance of getting a truly representative sample)
Proportional Stratified Sampling
The process of selecting a sample in such a way that identified subgroups in the population are represented in the sample in the same proportion in which they exist in the population
Qualitative Sampling
The process of selecting a small number of individuals for a study in such a way that the individuals chosen will be good key informants (reflective and thoughtful, good communicators, comfortable)
Non-proportional Stratified Sampling
The process of selecting equal-sized groups that do not align to subgroup sizes in the actual population
Triangulation
The process of utilizing multiple sources
Research Question
The purpose of research stated in the form of a question
Constant
a characteristic that takes on the same value for all individuals in a study
Constitutive Definition
a definition in which a word is defined by using other words
Ethnography
a description and interpretation of a cultural or social group system
data analysis section-research plan
a description of the technique or techniques that will be used to analyze the data collected during the study
design
a general strategy or plan for conducting a research study (includes the basic structure and goals of the study)
inductive hypothesis
a generalization based on specific observations that indicate certain patterns or associations among variables
Population (probability sampling)
a large group of individuals to whom the results of a study can be generalized
Cluster Sampling
a multistage sampling method in which clusters are randomly sampled and then a random sample of elements is taken from sampled clusters
Convenience Sample
a non-random sample in which you use a non-systematic selection method that often produces samples very unlike the population
Selection and definition of a problem
a question of interest that can be tested or answered through the collection and analysis of data
pilot study
a small-scale trial of a study conducted before the full-scale study to identify unanticipated problems or issues
database
a sortable, analyzable collection of units of information maintained on a computer
Sampling Frame
a specific list of sampling elements in the target population
meta-analysis
a statistical approach to summarizing the results of many quantitative studies that have investigated basically the same problem
Sample Size- Qualitative
a sufficient number of participants are needed to ensure that no new information is forthcoming from additional cases
instrument
a test or tool used for data collection
Source of Knowledge: Personal Experience
a way to acquire knowledge about and understand our world through our own experiences and observations
generalizable sampling
a well selected sample from a population that produces results applicable to other samples selected from the same population
What is the standard probability level?
a=.05
Constructs
abstraction that cannot be observed directly but only by their effects, such as creativity, anxiety, or intelligence
Unidimensionality
all items of an index or scale measure the same concept or have a common dimension
Random Assignment
all participants have an equal and independent chance of being assigned to each group
assumption
an assertion presumed to be true but not actually verified
Manipulation of the Treatments
The random assignment of participants to treatments in an experimental study
Sampling Ratio
The ration of the sample size to the size of the target population
Oral History
The researcher asks participants to tell their own story in their own way
Passive Observer
The researcher assumes no responsibilities in the setting
Casual-Comparative Research
The researcher attempts to determine the cause, or reason, for existing differences in the behavior or status of groups or individuals
Narrative Analysis
The researcher collects descriptions of events through interview s and observations and synthesizes them into narratives or stories, similar to the process of restorying.
How are narrative research and biography construction similar?
The researcher does not have direct access to observational data but must rely on primary data sources and secondary sources
Type II Error
The researcher incorrectly fails to reject the null hypothesis
Type I Error
The researcher incorrectly rejects the null hypothesis
Active Participant
The researcher is involve in the activities being observed (e.g teacher's aide)
Features of Quantitative Research
The researcher maintains control over contextual features, has little personal interaction with participants, and collects data in a non-interactive manner
Equality of Voice
The researcher must allow the participants voice to shine through without being judged
Control
The researchers' efforts to remove the influence of any variable, other then the independent variable, that may affect performance on the dependent variable
Generalizable
The results are applicable to other samples selected form the same population (Random sampling increases this)
Criterion
The score or outcome predicted by the predictor
Axial Coding
The second stage in grounded theory data analysis.
What does the researcher control about participants in an experimental study?
The selection and the assignment
Differential Selection of Participants
The selection of participants who have differences before the start of a study that may at least partially account for differences found in a posttest
Culture
The set of attitudes, values, concepts, beliefs, and practices shared by members of a group
Sampling interval
The size of the sample frame over the sample size, used in systematic sampling to select units
Shared Beliefs
The specific cultural conventions or statements that people who share a culture hold to be true or false.
Research Problem
The specific issue the researcher wishes to tackle
Standard Deviation
The spread of a set of scores around the mean
Coefficient of Determination
The squared correlation of the predictor and the criterion--higher coefficient means better prediction
Standard Error of the Mean
The standard deviation of the sample means; how far, on average, any sample mean would differ from the population mean SEX (X=Mean)
Specificity of Variables
The study is conducted with specific kind of participant, using specific instruments, at a specific time, and under specific circumstances; these need to be defined clearly
Ethnographic Research
The study of cultural patterns and perspectives of participants in their natural settings
Narrative Research
The study of how different humans experience the world around them; it involves a methodology that allows people to tell the stories of their "storied lives"
Narrative Research
The study of how humans experience the world around them, and it involves a methodology that allows people to tell the stories of their "storied lives"
Single-subject Experimental Designs
The study of people together as one group, rather than as individuals
Ethohistory
The study of the cultural past of a group of people.
Ethnographic Research
The study of the cultural patterns and perspectives of participants in their natural settings
Evaluation Research
The systematic process of collecting and analyzing data about the quality, effectiveness, merit, or value of programs, products, or practices
Response Set
The tendency of an individual to respond in a particular way to a variety of instruments
Statistical Regression
The tendency of participants who score highest on a test to score lower on a second, similar test
Predictor
The test that "predicts" a future score or outcome
What is the concept underlying ANOVA?
The total variation or variance of scores can be divided into two sources--variance between groupw and variance within groups
Dependent variable
The ultimate or outcome variable of interest.
Common Sense
The use of human reasoning as a basis for decisions.
Independent Variable
The variable being manipulated
Criterion
The variable being predicted
Independent variable
The variable in an experimental study over which the researcher has control; assumed to be the cause of something else.
Dependent Variable
The variable that changes as a result of the manipulation of the independent variable
Predictor
The variable used to predict
Norms
The written and unwritten rules that specify appropriate behavior.
Ethnography
The written report, the product, of an ethnographic study
Problems With Educational Inquiry
There are many ethical concerns and responsibilities that may skew results, and much research may be done beforehand
Comparative Studies
These studies investigate the relationship of one variable to another by examining differences on the dependent variable between two groups of participants. Criteria for evaluating these studies -Participants and instruments are described completely -Criteria for identifying the different groups is clearly stated -No inferences are made about causation -Graphs and images depict the results accurately
What is one useful characteristic of sampling errors?
They are usually normally distributed, so the mean of all sample means will yield a good estimate of the population mean.
Self-regulation
Thinking about thinking (self examination) and making appropriate corrections (self correction)
Scientific Method
an orderly process entailing a number of steps : recognition and definition of a problem; formulation of hypothesis; collection of data; analysis of data; and statement of conclusions regarding confirmation or disconfirmation of the hypothesis
The Scientific Method
an orderly process entailing a number of steps: recognition and definition of a problem; formulation of hypotheses; collection of data; analysis of data; and statement of conclusions regarding confirmation or dis-confirmation of the hypotheses
theory
an organized body of concepts, generalizations, and principles that can be investigated
action research
applied research focused on solving practicioners local problems
Single-subject experimental designs
are those used to study the behavior change that an individual or group exhibits as a result of some intervention or treatment
Deductive Reasoning
arriving at specific conclusions based on general principles, observations, or experiences (i.e. the text includes a chapter on sampling, which means the text is a research book)
refereed journal
articles reviewed by a panel of experts in the field and are thus seen as more scholarly and trustworthy than articles from non-refereed or popular journals
Experimental Research
at least one independent variable is manipulated, other relevant variables are controlled, and the effect on one or more dependent variables is observed
Causal-comparative research
attempts to determine the cause, or reason, for existing differences in the behavior or status of groups of individuals.
National Research Act of 1974
authorized the creation of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research - charged with the development of ethical code and guidelines for researchers.
Simple Random Selection (Probability)
bias free- fishbowl technique, every individual's name is placed in a hat or bowl and names are drawn
Dichotomous Variable
categorical variable that has only two classes
Nominal Measurement Scale
categories: Race, gender, types of schools (e.g., public, private, parochial)
Correlational Quantitative Design
To ascertain the extent to which two or more variables are statistically related
Descriptive Quantitative Design
To describe the current status of a variable of interest to the researcher
What is the goal of ethnographic research
To describe, analyze, and interpret the culture of a group, over time, in terms of the group's shared beliefs, behaviors, and language
Experimental Quantitative Design
To establish cause and effect relationships between variables
Bracket
To suspend your preconceptions or learned feelings about a phenomenon.
Mean
Total of the scores divided by the number of scores.
operational definitions
clearly describe variables in measurable ways and clarify important terms in a study so that all readers understand the precise meaning the researcher intends
Basic Research
collection and analysis of data to develop or enhance theory - helps provide a solid foundation
Mixed Methods Research
combining quantitative and qualitative research in different ways, with each approach adding something to the understanding of the phenomenon
grounded theory
conducted to generate or discover a theory or schema that relates to a particular environment
primary source
contains firsthand information such as an original document or description of a study written by the person who conducted it
abstracting
creating an organization system of potential references specifically for the literature review by re-writing the abstracts of each source with the most pertinent information related to the research problem to determine the most relevant resources
Operational Definition
definition that specifies the procedure or operation to be followed in producing or measuring a concept
deductive hypothesis
derrived from theory and provides evidence that supports, expands, or contradicts the theory
Procedure section-research plan
describes all the steps in collecting the data from beginning to end in the order in which they will occur
quantitative topic statement
describes the variables of interest, the specific relations among those variables, and ideally, important characteristics of the participants
Formulative Evaluation
designed to inform and improve a program while it is being developed or implemented
research plan
detailed description of a study proposed to investigate a given problem
Survey Research
determines and reports the way things are; it involves collecting numerical data to test hypotheses or answer questions about the current status of the subject of study.
Examples of Applied Research
developing a 7th grade social studies curriculum around a problem solving approach to learning
Inductive Reasoning
developing generalizations based on observation of a limited number of related events or experiences
Evaluation research
directed toward making decisions about the effectiveness or desirability of a program
Median
divide the scores, rank order from top to bottom, in half 50% of the scores are above the median, 50% below
quasi-experimental
does not have random assignment
Dichotomies
either/or terms
Population
entire group or aggregate of people having one or more common characteristics e.g., all college students, all adolescents, all student at New Haven High School
Ratio measurement scales
equal intervals and an absolute zero (0): Height, weight, time
Interval measurement Scale
equal intervals between numbers on the scale e.g. Test scores, achievement levels
qualitative topic statement
eventually narrow as the researcher learns more about the research context and its inhabitants-more precise statements appear in the research report
Test Content (Validity evidence)
evidence of the extent to which items on a test are representative of the larger domain of content or items from which they are drawn
Relationships w/ other variables (Validity evidence)
evidence of the extent to which scores from an instrument are related to similar as well as different traits
Internal Structure (Validity evidence)
evidence of the extent to which the relationships between items and parts of the instrument are consistent with those reflected in the theoretical basis of the instrument or its intended use
causal-comparative
examines whether a naturally occurring intervention affects an outcome of interest. such as student performance.
examples of applied research
examining the effectiveness of a computer based algebra program developed around a mastery learning approach
scientific hypothesis
explanation for the occurrence of certain behaviors, phenomena, or events as a way of predicting the results of a research study and then collects data to test that prediction
Ethnography
form of qualitative research that aims for a holistic picture of a cultural group; it uses in-depth interviewing and prolonged participant observation
Basic Interpretative Studies
form of qualitative research that provides a descriptive account targeted to understanding a phenomenon using data that may be collected in a variety of ways.
Narrative Inquiry
form of qualitative research, also called biography, life stories, and life narratives, that provides accounts of a person's experiences.
basic research
formulates and refines theories
Qualitative Research
generic term for a variety of research approaches that generally gather non-numeric data to study phenomena, without predetermined hypothesis.
Ways to Increase Response Rate
give specific deadline, use individually signed letters, include stamped addressed return envelope, contact participants in advance, promise anonymity or confidentiality, follow up with reminders, interview non-responders by phone (small, random number)
Scatterplot
graphic depictions of correlations
purpose of scientific method
guide researchers in planning, conducting, and interpreting research studies
Experimental quantitative design
important characteristic of this is that the researcher manipulates the independent variable and controls extraneous variables
applied research
improves practice and solves practical problems
Action Research
in education is any systematic inquiry conducted by teachers, principals, school counselors, or other stakeholders in the teaching-learning environment to gather information about ways in which their particular school operate, the teachers teach, and the students learn.
case studies
in-depth study of a single or few programs defined in terms of time and place.
Descriptive
includes studies that provide simple information about the frequency or amount of something
Cluster Sampling (Probability)
intact group of people are selected for the sample rather than individuals, not bias free, e.g., teachers in several schools.
Correlational
investigate relationships among two or more variables
Correlational Research
involves collecting data to determine whether, and to to what degree, a relation exists between two or more quantifiable variables.
Interpretation
involves determining meaning and significance and is aided by clear formation of categories
Evaluation Research
involves determining the worth, merit, of quality of an evaluation object
Inductive Reasoning
involves developing generalizations based on observation of a limited number of related events or experiences.
Deductive Reasoning
involves essentially the reverse process to inductive reasoning - arriving at specific conclusions based on general principles, observations, or experiences
Variable
is a placeholder that can assume any one of the range of values; for example, intelligence, height, and test score are variables.
Case Study Research
is a qualitative research approach to conducting research on a unit of study or bounded system (i.e. an individual teacher, a classroom, or a school)
Applied Research
is conducted for the purpose of applying or testing a theory to determine it usefulness in solving practical problems.
Basic Research
is conducted solely for the purpose of developing or refining theory
Quantitative Research
is the collection and analysis of numerical data to describe, explain, predict, or control phenomena of interest including such approaches as; Survey Research, Correlational Research, Causal-Comparative Research, Experimental Research and Single-Subject Research
Qualitative Research
is the collection, analysis, and interpretation of comprehensive narrative and visual (i.e nonnumerical) data to gain insights into a particular phenomenon of interest including; Narrative Research, Ethnographic Research, Case Study Research
P-value
is the probability (p) that a result could have been produced by chance if the null hypothesis were true.
Research & Development
is the process of researching consumer needs and then developing products to fulfill those needs.
Narrative Research
is the study of how different humans experience the world around them; it involves a methodology that allows people to tell the stories of their "storied lives"
Ethnographic Research
is the study of the cultural patterns and perspectives of participants in their natural settings
Evaluation Research
is the systematic process of collecting and analyzing data about the quality, effectiveness, merit, or value of programs, products, or practices - focuses mainly on decisions - decisions about programs, products, and practices
Informed Consent
making sure that research participants enter the research of their own free will and with understanding of the nature of the study and any possible dangers that may arise as a result of their participation.
Experimental Treatment
manipulated variable or independent variable
MANOVA
multivariate analysis of variance--needs larger sample size with meaningful variation among the variables
Quota Sampling (Non-Probability)
non-random sampling representative of a larger population - Used when the researcher cannot use probability sampling procedures but does want a sample that is somewhat representative of the population - Similar to stratified sampling with the exception that the subjects are selected non-randomly
O.C.
observer's comment
Four Data Collection Techniques
observing, interviewing, questionnaires, and examining records
Ordinal Measurement Scale
ordered categories:finishing position in a race, grade levels
Positivism
philosophy of research characterized by objective inquiry based on measurable variables, believing that science should be primarily concerned with the explanation and the prediction of observable events. underlies quantitative research
Statistics
procedures that summarize and analyze quantitative data
applied research
provides the data to help support, guide, and revise the development theory
Basic Research
provides the theory that produces the concepts for solving educational problems
Single subject quantitative designs
purpose of this is to investigate the cause an effect relationships with a sample of one
Case Study
qualitative examination of a single individual, group, event, or institution
Phenomenological Studies
qualitative research that focuses on understanding the meaning events have for individuals in particular situations.
guiding hypothesis
qualitative researchers that generate new hypotheses as a result of their studies rather than solidifying the intention of the research beforehand
Correlation
refers to a quantitative measure of the degree of correspondence
Merit
relates to a program accomplishing what it was supposed to accomplish
Worth
relates to the value attatched to a program by those using it
replication
repetition of a study using different subjects to retest its hypothesis
Variable
representation of a construct that takes on a range of values.
orientational research
research done for the purpose of advancing an idealogical position
Quasi-Experimental Research
research in which the investigator can control the treatment and the measurement of the dependent variable but cannot control assignment of the subjects to treatment.
Experimental Research
research in which the investigator manipulates one or more independent variables and observes the effect on one or more dependent variables
Content Analysis
research method applied to written or visual materials to analyze characteristics of the material
Basic Research
research that aims to obtain empirical data that can be used to formulate, expand, or evaluate a theory rather than to solve a practical problem
Applied Research
research that aims to solve an immediate practical problem
Descriptive Research
research that asks questions about the nature, incidence, or distribution of variables; it involves describing but not manipulating variables.
Correlation Research
research that attempts to determine the extent and the direction of the relationship between two or more variables.
Applied Research
research that is focused on answering practical questions to provide relatively immediate solutions -provides answers to real world solutions
Normal frequency distribution shape
scores are equally distributed around the middle
Convergent Evidence (Validity evidence)
scores correlate with measures of the same thing being measured - BMI and Body Fat tests to measure fat and fat free mass
Discriminate Evidence (Validity evidence)
scores do not correlate with measures of something different than that being measured - Educational level and Poverty
Outlier scores (frequency distribution shape)
scores that distort findings because they are so different from the other scores in the sample
secondary source
secondhand information such as a brief description of a study written by someone other than the person who conducted it
Convenience Sampling (Non-Probability)
selecting a participant or group of participants based on their availability to the researcher -The major concern is the limited generalizability of the results from the sample to any population
Typical Case (purposeful techniques)
selecting a representative participant
Extreme Case (purposeful techniques)
selecting a unique or atypical participant
Criterion (purposeful techniques)
selecting individuals with certain important characteristics
Snowball (purposeful techniques)
selecting participants from recommendations of other participants
Maximum Variation (purposeful techniques)
selecting participants to represent extreme cases
Critical Case (purposeful techniques)
selecting the most important participants to understand the phenomena being studied
Purposeful Sampling (Qualitative)
selection of particularly informative or useful participants (think focus groups) - Typically selects information-rich participants who are studied in-depth
action research
solve a specific classroom or school problem, improve practice, or help make a decision at a single local site
Systematic Sampling (Probability)
some "system" is applied for the selection of the subjects, e.g., select every fifth person.
limitation
some aspect of the study that the researcher cannot control but believes may negatively affect the results of the study
non-directional hypothesis
states simply that a relation or difference between variables exists
directional hypothesis
states the expected direction of the relation or difference between variables
null hypothesis
states there is no significant relation or difference between variables
Descriptive Statistics
statistical procedures that summarize a set of numbers in terms of central tendency, variation, or relationships
True experimental
subjects have been randomly assigned to different groups
Historical Research
systematic attempt to establish facts and arrive at conclusions about the past
Document Analysis
systematic examination of documents to investigate specific topics or themes.
Quantitative Research
the collection of numerical data to explain, predict, and or control phenomena of interest
Qualitative Research
the collection, analysis, and interpretation of narrative and visual data in order to gain insights into something of interest.
Sample
the collective group of participants from whom data are collected - member of the population, representative of that population. possess similar characteristics of the pop. sample statistics = pop. parameters.
Research Purpose
the degree of direct applicability of research to educational practices and settings
generalizabilty
the degree to which the results of research can be applied, usually to the "real world"
Correlation coefficient
the degree to which two variables are related is expressed
Research and development
the development of a "Smart Board" to enhance a teachers use of technology in the classroom
Sampling Bias
the difference between the "observed" and "true" results that is attributed to the sampling mistakes of the researcher
Range Variability
the difference between the highest and lowest scores
Validity
the extent to which inferences are appropriate, meaningful, and useful...Does the 'measure', measure what it should? -Refers to the interpretation of the results - A matter of degree - Specific to a particular use or interpretation - A unitary concept - Involves an overall evaluative judgment
Predictability (Validity evidence)
the extent to which test scores predict performance on a criterion variable - SAT Scores
goal of Educational Research
to describe, explain, predict, or control educational phenomena
Causal-Comparitive Quantitative Design
to explore relationships among variables that cannot be actively manipulated or controlled by the researcher
Survey Research
type of non experimental research in which investigators ask questions about peoples' beliefs, opinions, and behavior.
Ex Post Facto Research
type of research that attempts to determine the causes for, or the consequences of, differences that already exist in groups of individuals.
single-subject
use the ideas of an experiment with a single person or a few individuals
Categorical Variable
variable that differs in kind, not in degree or amount
Dependent Variable
variable that is a consequence of or dependent on an antecedent (independent) variable
Independent Variable
variable that is antecedent to the dependent variable
Continuous Variable
variable whose measure can take an infinite number of points within a range
Theoretical Validity
How well the report relates the phenomenon under study to a broader theory
Experimental Group
The group that receives the new treatment
Protocol
The guide the researcher uses to structure an interview
Empirical Hypothesis
The hypothesis stated in terms of specific measures of variables
Reactive Arrangements
(Participant effects) threats to validity due to the way a study is conducted and the feelings of participants
A Posteriori
(Post Hoc) Comparisons of multiple groups planned after the fact
Quantitative Research Question
An interrogative sentence that discusses the relationship between two variables
Essence
An invariant structure of the experience.
Convenience Sampling
(Accidental sampling or Haphazard Sampling) The process of including whoever happens to be available at the time (volunteer or studying existing groups) [Students enrolled in a class]
Mortality
(Attrition) the loss of study participants over time (can impact the characteristics of groups); occurs more often in the experimental groups because more is being asked of them
Performance Assessment
(Authentic or Alternative Assessment) Emphasizes student process
Accessible Population
(Available population) The population from which the the researcher can realistically select subjects
Nominal Variable
(Categorical Variable) Values include two or more named categories (man/woman; eye color)
External Validity
(Ecological Validity) The degree to which study results are generalizable to groups outside the experimental setting
Critical Action Research
(Emancipatory Action Research); Based on a body of critical theory
Equivalence
(Equivalent-Forms Reliability) Are two forms the same?
Family Educational Rights and Privacy Act of 1974
(FERPA or the Buckley Amendment) Specification that data that identify a student may not be made available unless written permission is acquired from the student (if of legal age) or a parent or legal guardian
Compensatory Rivalry
(John Henry Effect) Members of a control group feel threatened or challenge by being in competition and perform way beyond normal expectation
Purposive Sampling
(Judgement sampling) The process of selecting a sample that is believed to be representative of a given population [Studying exceptional high schools]
Structural Equation Modelin
(LISREL) Clarifies direct and indirect interrelations among variables relative to a given variable
Ethical Guideposts
1) A researcher should have an ethical perspective with regard to the research that is very close to his or her personal ethical position. 2) Informed consent should take the form of a dialogue that mutually shapes the research and the results. 3) Researchers should also think beyond the methods they plan to use; they must identify broader social principles that are integral parts of who they are as researchers and as contributing members of the communities in which they live. 4) Qualitative researchers are morally bound to conduct researcher in a manner that minimizes potential harm to those involved in the study. 5) Even though an action may bring about good results, it is not ethical unless that action also conforms to ethical standards such as honesty and justice. 6) The qualitative researcher must remain attentive to the relationship between the researcher and the participants, a relationship determined by "roles," status, language, and cultural norms.
Differences Between Casual-Comparative and Experimental Studies
1) Casual-comparative studies are done ex post facto, while experimental studies are done proactively (controlling variables). 2) Variables cannot be controlled and manipulated with casual-comparative studies, but they are in experimental studies. 3) In casual-comparative research, the "independent" variable is better called the grouping variable. 4) In casual-comparative research, the groups are already formed, while experimental studies form the groups actively as part of the study.
Differences Between Casual-Comparative and Correlational Studies
1) Casual-comparative studies attempt to find a cause, but correlational studies do not. 2) Casual-comparative studies usually involve two or more groups, but correlational studies usually involve two or more variables but only one group. 3) Casual-comparative studies focus on the differences between groups, but correlational studies involve relations among variables.
What are the three types of experimental comparisons?
1) Comparison of two different approaches (A versus B) 2) Comparison of a new approach to the existing approach (A versus no A) 3)Comparison of different amounts of a single approach (a little of A versus a lot of A)
Three Conceptual Ares That Focus on Tangible Behaviors
1) Cultural Orientation 9where people under study are situated in terms of physical space and activities) 2) Cultural Know-How (how a group goes about its daily activities) 3) Cultural Beliefs (why a group does what it does)
4 Characteristics of Critical Research
1) Democratic (participation of all people) 2) Equitable (acknowledging people's equality) 3) Liberating (Providing freedom from oppressive, debilitating conditions) 4) Life enhancing (Enabling expression of people's full human potential)
Qualitative Research Process
1) Identify a topic 2) Review the literature 3) Selecting participants 4) Collecting data 5) Analyzing and interpreting data 6) Reporting and evaluating the research
Steps in a Literature Review
1) Identify and make a list of keywords to guide your literature search 2) Using your keywords, locate primary and secondary sources that pertain to your research topic. 3) Evaluate your sources for quality 4) Abstract your sources 5) Analyze and organize your sources using a literature matrix 6) Write the literature review
Dialectic Action Research Spiral
1) Identify area of focus 2) Data collection 3) Data analysis and interpretation 4) Action Planning
Questionnaire Suggestions
1) Include only items that relate to the objectives of the study. 2) Collect demographic information about the sample if you plan to make comparisons between different subgroups. 3) Focus each question on a single concept. 4) Define or explain ambiguous terms (many/several). 5) Include a point of reference to guide respondents in answering questions. 6) Avoid leading questions, which suggest that one response may be more appropriate than another. 7) Avoid sensitive questions to which the respondent may avoid or not answer honestly. 8) Don't ask a question that assumes a fact not necessarily true.
Characteristics of a Good Topic
1) Interesting 2) Researchable 3) Theoretical or Practical Significance 4) Ethical 5) Manageable for You
Two Common Problems in Experimental Studies
1) Lack of sufficient exposure to treatments 2) Failure to make treatments substantially different from each other
Time sequence
An investigative technique that looks at the time that variables occur in relation to each other to determine plausible cause and effect
Guidelines for Constructing a Questionnaire
1) Make the questionnaire attractive and brief. 2) Know what information you need and why. 3) Include only items that relate to your study's objectives. 4) Collect demographic information, if needed. 5) Focus items on a single topic or idea. 6) Define or explain ambiguous terms. 7) Word questions as clearly as possible. 8) Avoid leading questions. 9) Avoid or carefully word items that are potentially controversial or embarrassing. 10) Organize items from general to specific. 11) Use examples if item format is unusual. 12) If using open-ended items, leave sufficient space for respondents to write their responses. 13) Try to keep items and response options together. 14) Subject items to a pretest review of the questionnaire. *15) Don't put the most important items at the end.*
Abstracting Steps
1) Read the abstract or summary to determine the relevance of the article to your problem 2) Skim the article, making mental notes of the main points of the study 3) Write a complete bibliographic reference for the work 4) Classify and code the article according to some system, and then add the code to the database entry or index card 5) Abstract or summarize the reference and its key points 6) Indicate thoughts that come to your mind (questions, disagreements, etc.) 7) Indicate any statements that are direct quotations or personal reactions
Ethical Issues in Qualitative Research
1) Research plans evolve and change as the researcher's immersion in and understanding of the research setting grow. 2) Qualitative researchers are personally engaged in the research context. 3) Researchers might have to make difficult choices involving ethics.
3 Types of Reconnaissance
1) Self-Reflection 2) Description (Describe the situation you want to change-who, what, where, when) 3) Explanation (why)
4 Criteria for Identifying an Area of Focus
1) Should involve teaching and learning and focus on your own practice 2) Something within your locus of control 3) Something you feel passionate about 4) Something you would like to change or improve
Why doesn't educational research affect teaching?
1) Teachers do not find research persuasive or authoritative 2) Research has not been relevant to practice and has not addressed teachers' questions 3) Findings from research have not been expressed in ways that are comprehensible to teachers 4) The education system itself is unable to change, or conversely, it is inherently unstable and susceptible to fads
Possible Concerns with Casual-Comparative Studies
1) The researcher has limited control, so caution must be used when interpreting results. 2) Sometimes it is not possible to determine which variable came first and influenced another (reversed causality).
Questions to Evaluate Literature Sources Critically
1) What was the problem statement of the study? 2) Who was studied? 3) Where was the source published? 5) When was the research conducted? 6) How was the study conducted?
Five Characteristics That Determine Category of Research Approach
1) Who authored the account? 2) The scope of the narrative 3) Who provides the story? 4) The kind of theoretical/conceptual framework that has influenced the study 5) Whether or not all these elements are included in one narrative
Limitations of the Scientific Method
1) cannot answer all questions; 2) can never capture the full richness of the individuals and the environments under study; 3) measuring instruments always have some degree of error
Tabulate Responses
1) code open-ended question responses according to provided patterns 2) include total sample size and the overall percentage of returns along with response rate for each item 3) simplest way is to present the percentage of respondents who selected each alternative 4) item by item descriptions or cluster responses for reporting
Scoring Procedures
1) convert behavior data into numeric system 2) provide a scoring key in the manual 3) score consistently with one criterion 4) recheck some tests (25%) 5) have at least two people independently score open ended items
Two Categories of Threats to External Validity
1) generalizing to whom (threats to population validity) 2) generalizing to what (threats to ecological validity)
Normally Distributed (Normal Curve) Properties
1)50% of scores are above and 50% are below the mean 2)mean, median, and mode are the same 3)most scores are near the mean 4)34.13% of scores fall one standard deviation above the mean and the same below; 99% fall somewhere between 3 standard deviations of the mean (1 sd=68%, 2 sd=95%)
Scientific Method
An orderly process that includes recognition and definition of a problem, formulation of a hypothesis, collection of data, analysis of data, and statement of conclusions
Scientifically based research
1. logical evidence-based chain of reasoning 2. Methods appropriate to the question posed 3. Design and instruments that yield reliable findings 4. Data and their analysis sufficient to support conclusions 5. Info to evaluate the generalization of findings 6. Accessibility of data for reanalysis and replication 7. Use of designs that reduce plausible competing explanations
Collaborative action research
Action research that is designed and implemented by collaborative teams.
Experimenter Bias Effect
Active experimenter effect caused by the researcher's expectations of study results
Analysis of Covariance
Adjusting scores on a dependent variable to account for initial differences (e.g. Group X had a higher starting score than Y, so the researcher statistically adjust the scores so the two groups can be fairly compared)
What types of instruments tend to have low validities?
Affective Instruments
Advantage of a Relationship Study
All data may be collected in a short period of time
Exhaustive Attributes
All units fit into some category of a variable
Path Analysis
Allows us to see the relations and patterns among a number of variables (visual paths)
Qualitative studies
Almost exclusive reliance on purposeful techniques (sampling procedure)
Construct
An abstraction that cannot be observed directly; a concept invented to explain behavior (intelligence, personality)
Systematic Sampling
An approximation to random sampling which you select one in a certain number of sample elements; the number is from the sampling interval
Subtest
An assessment in a specific area done as part of a test battery
Etic Perspective
An external, social scientific view of reality.
Curvilinear Relation
An increase in one variable causes an increase or decrease in another to a certain point, at which the response reverses (e.g. car value decreases until the car becomes an antique)
Linear Relation
An increase or decrease in one variable is associated with a corresponding increase or decrease in another variable
Life-World
An individual's inner world of immediate experience.
Criterion-Referenced Scoring
An individual's performance is compared to a pre-determined, external standard, rather than to the performance of others
X
An individual's score on the predictor variable
Selection-Maturation Interaction
An interactive effect of multiple issues; participants selected into groups mature at different rates (e.g. A sub teaches one class but not the other);
Qualitative Research Question
An interrogative sentence that discusses some problem or phenomenon to be explored
Structured Interview
Formal set of questions
Test
Formal, systematic, usually paper-and-pencil method for gathering information about people's cognitive and affective characteristics
Types of Descriptive Statistics
Frequencies, measures of central tendency, measures of variability, measures of relative position, measures of relationship
Inductive Hypothesis
Generalization from specific observations
How do you choose which measure of central tendency to use?
Good judgement-avoid lying with statistics
Focus Group
Group interview in which you try to collect shared understanding
3 Levels of Action Research
Group of Teachers (most common), Single School, Individual Teacher
Multiple Indicators
Having several different specific measures that to indicate the same concept
Inferential Statistics and Genereralization
Help researchers to know whether they can generalize to a population of individuals based on information obtained from a limited number of research participants
Ranked Items
Highest to lowest
Postmodernism
Historical intellectual movement that constructs its self-image in opposition to modernism; postmodernism emphasizes the primacy of individuality, difference, fragmentation, flux, constant change, lack of foundation for thought, and interpretation.`
Poststructuralism
Historical intellectual movement that rejects universal truth and emphasizes differences, deconstruction, interpretation, and the power of ideas over people's behavior.
Eight Main Threats to Internal Validity
History, Maturation, Testing, Instrumentation, Statistical Regression, Differential Selection of Participants, Mortality, Selection Maturation Interaction
Analytical Research
History, legal or concept analysis
Example of action research
How can disciplinary policies be enforced consistently in our school?
Variability
How different the scores are
Relationship
How do two sets of scores relate to one another
Examples of a Descriptive Quantitative Design
How many students drop out of school in Louisiana? To what extent are elementary teachers using math manipulatives?
Descriptive Statistics
How often or frequent an event or score occured
Level of Measurement
The degree a measure is refined or precise
Sampling Error
The degree to which a sample deviates from a population
Face Validity
The degree to which a test appears to measure what it claims to measure
Predictive Validity
The degree to which a test can predict how well an individual will do in a future situation
Reliability
The degree to which a test consistently measures whatever it is measuring
Construct Validity
The degree to which a test is measuring the intended hypothetical construct
Content Validity
The degree to which a test measures an intended content area
Internal Validity
The degree to which observed differences on the dependent variable are a direct result of manipulation of the independent variable, not some other variable
Concurrent Validity
The degree to which scores on one test are related to scores on a similar, preexisting test administered in the same time frame or to some other valid measure available at the same time
Measures of Relationship
The degree to which two sets of scores are related
Phenomenology
The description of one or more individuals' consciousness and experience of a phenomenon.`
Quantitative studies
The desired use of probability sampling due to the ability to generalize the results to the larger population. Frequent use of non-probability techniques - particularly convenience sampling - due to access, time, resource, or financial constraints (sampling procedure)
Standard Error
The deviation of predicted scores
Range
The difference between the highest and lowest scores in a distribution
Ethnography
The discovery and comprehensive description of the culture of a group of people.
Confounded
The effects of the independent variable are so intertwined with those of extraneous variables that it becomes difficult to determine the effects of each
Descriptive
The end result of the case study, the narrative, includes "thick description" of the phenomenon that was the focus of the case study
Consequential Validity
The extent to which an instrument creates harmful effects for the user
Generalization
The extent to which findings apply to other people, setting, contexts, or materials
Descriptive Validity
The factual accuracy of an account
Selective Coding
The final stage in grounded theory data analysis.
Open Coding
The first stage in grounded theory data analysis.
Measurement Validity
The fit between a concept and how it is measured
Research question
The fundamental question inherent in the research topic under investigation.
Goal of Grouping in Casual-Comparative Research
The goal is to have groups that are as similar as possible on all relevant variable except the grouping variable
Scientific Method
The goal of this is to explain, predict, and control phenomena *Involves the acquistion of knowledge *Use of this method is more effcient and reliable than any other source of knowledge
Experimental group
The group of participants in an experimental study that is exposed to a new or an innovative treatment or another type of condition.
Control group
The group of participants in an experimental study that receives the "standard" treatment.
Population
The group of people of primary interest in a research study, although not necessarily studied in its entirety; the group about which conclusions are drawn.
Control Group
The group that receives a different treatment from the experimental group (not "no treatment")
Educational Research
the formal, systematic application of the scientific method to the study of educational problems
Research
the formal, systematic application of the scientific method to the study of problems
Sampling Frame (probability sampling)
the group to whom the researcher has access and from which the actual sample will be drawn- often the sampling frame and the target population are different
Target Population (probability sampling)
the group to whom the results are intended to be generalized- often the sampling frame and the target population are different
target population
the ideal population the researcher would like to generalize the study results
Sample
the individuals selected from a population for a study
Margin of Error (probability sampling)
the interval (95% probability) within which the true population lies
population
the larger group from which the smaple will be selected in research
mode
the most frequently occurring score
Research Method
the overall strategies followed to collect and analyze data
Research Design
the plan and structure of research to provide a credible answer to a research question
Stratified Random Sampling (Probability)
the population is divided into strata or subgroups- selection is based on proportionally selecting subjects from each group -It is the best way to get representation, e.g., group divided by race: White, African Am., Hispanics.
Source of Knowledge: Reasoning
the process of using logical thought to reach a conclusion
accessible population (available population)
the realistic population from which the researcher can select subjects
research hypothesis
the researcher's prediction of the research findings, a statement of the researcher's expectations about the relations among the variable in the research topic
Median
the score above and below which one-half of the scores occur
negatively Skewed Frequency Distribution Shape
the set of scores is characterized by a large number of high scores and a small number of low scores
Positively skewed frequency distribution shape
the set of scores is characterized by a large number of low scores and a small number of high scores
review of related literature
the systematic identification, location, and analysis of documents containing information related to the research problem
metaphorical thinking
the use of the metaphor in reasoning
Grounded Theory
theory derived inductively from the data collected rather than from a priori ideas or theories
