603 Final Exam

Discuss different types of observations and interviews. focus group interview:

An interview conducted with a group in which respondents hear the views of each other.

Discuss different types of observations and interviews. participant observation:

Observation in which the observer actually becomes a participant in the situation to be observed.

Discuss different types of observations and interviews. naturalistic observation/nonparticipant observation:

Observation in which the observer controls or manipulates nothing and tries not to affect the observed situation in any way.

Key terms related to ethnographic research: crystallization :

Occasions, especially in ethnography, when different kinds of data "fall into place" to make a coherent picture.

Threats to internal validity in experimental research

Subject-Characteristics attitude of subjects Insturment decay implementation Data collector characteristics Data collector bias testing regression history maturation Mortality Location

Threats to internal validity of correlational research

Subject characteristics Mortality Location Instrument decay Testing History Data collector characteristics Data collector bias SMITH LD

Evaluation research:

aims to improve the object or program being evaluated, usually by strengthening its delivery, implementation, and outcomes.

Describe approaches to qualitative research. ethnography:

appropriate for behaviors that are best understood by observing them within their natural settings. The key techniques in all ethnographic studies are in-depth interviewing and highly detailed, almost continual, ongoing participant observation of a situation. A key strength of ethnographic research is that it provides the researcher with a much more comprehensive perspective than do other forms of educational research.

Threats to internal validity in randomized posttest-only control group, randomized pretest-postest control group, randomized solomon four-group, randomized postest-only control group with matched subjects, and matching-only pretest-posttest control group are:

location data collector characteristics data colector bias attitude of subjects implementation LAIDD

factorial with and without randomization are affected by

location data collector characteristics and bias attitude of subjects implementation LAID

Counterbalanced design is affected by

location data collector characteristics and bias implementation LID

Threats to the internal validity of survey research

location instrumentation instrument decay mortality MIIL

Describe approaches to qualitative research. biographical study:

tells the story of the special events in the life of a single individual.

Describe approaches to qualitative research. case study:

A detailed study of one or (at most) a few individuals or other social units, such as a classroom, a school, or a neighborhood. It can also be a study of an event, an activity, or an ongoing process.

Discuss different types of observations and interviews. interview:

A form of data collection in which individuals or groups are questioned orally. One purpose of interviewing the participants in a qualitative study is to find out how they think or feel about something. Another purpose is to provide a check on the researcher's observations. The most effective characteristic of a good interviewer is a strong interest in people and in listening to what they have to say.

Discuss different types of observations and interviews. retrospective interview:

A form of interview in which the researcher tries to get a respondent to reconstruct past experiences.

phenomenology:

A form of qualitative research in which the researcher attempts to identify commonalities in the perceptions of several individuals regarding a particular phenomenon. (lived experiences!)

grounded theory:

A form of qualitative research that derives interpretations inductively from raw data with continual interplay between data and emerging interpretations. (try to create a theory grounded on data)

Discuss different types of observations and interviews. structured interview:

A formal type of interview, in which the researcher asks, in order, a set of predetermined questions.

Describe what frequency tables, bar graphs, and pie charts are.

A frequency table/distribution is a tabular method of showing all of the scores obtained by a group of individuals. A frequency distribution is a two-column listing, from high to low, of all the scores along with their frequencies. A grouped frequency distribution is a frequency distribution in which scores are grouped into equal intervals. A bar graph is a graphic way of illustrating differences. It shows bars spread apart. This is different from a histogram which is used to present "continuous data" (with bars back to back). A pie chart is a graphic method of displaying the breakdown of data into categories.

Describe approaches to qualitative research. phenomenological study:

A researcher studies an individual's reactions to a particular phenomenon. He or she attempts to identify the commonalities among different individual perceptions.

Discuss different types of observations and interviews. semistructured interview:

A structured interview, combined with open-ended questions.

Describe action research and how it can be implemented in a variety of settings.

Action research is conducted by a teacher, administrator, or other education professional to solve a problem at the local level. Each of the specific methods of research can be used in action research studies, although on a smaller scale. A given research question may often be investigated by any one of several methods. participants have the authority to make decisions, want to improve their practice, are committed to continual professional development, and will engage in systematic inquiry. Practical action research addresses a specific local problem. Participatory action research, while also focused on addressing a specific local problem, attempts to empower participants or bring about social change. Action research studies suffer especially from the possibility of data collector bias, implementation, and subject attitudinal threats. Most others can be controlled to a considerable degree. DIS How it can be implemented: 1) Using a comparison-group experiment: randomly assign students to one treatment and the other students to a control. Look at the difference. 2) Single subject experiment: see what the effects of some treatment are on a single student (ex. a disruptive student receives the "time-out" treatment and we look at the change if any) 3) Survey: determining how students feel about something (what students like about school) 4) Content analysis: looking at certain categories of items in a text (checking for bias in a textbook) 5) Correlational study (ex. predicting what students are going to struggle in algebra) 6)Ethnographic: ex. looking at how teachers teach music

What threats affect one-shot and one-group designs

All threats

reliability in qualitative research

Consistency of inferences over time, location, and circumstances. Compare one informant's description of something with another informant's description of the same thing. Efforts to ensure reliability and validity include use of proper vocabulary, recording questions used as well as personal reactions, describing content, and documenting sources.

Explore how content-analysis can be used to conduct educational research.

Content analysis is an analysis of the contents of a communication. A method of studying human behavior indirectly by analyzing communications, usually through a process of categorization. There are several reasons to do a content analysis: to obtain descriptive information of one kind or another; to analyze observational and interview data; to test hypotheses; to check other research findings; and/or to obtain information useful in dealing with educational problems. Reliability in content analysis: is commonly checked by comparing the results of two independent scorers (categorizers). Validity: can be checked by comparing data obtained from manifest content to that obtained from latent content. A common way to interpret content analysis data is by using frequencies (i.e., the number of specific incidents found in the data) and proportion of particular occurrences to total occurrences. Content analysis is often used with historical and ethnographic research.

Key terms related to ethnographic research: triangulation:

Cross-checking of data using multiple data sources or multiple data-collection procedures.

descriptive statistics

Data analysis techniques that enable the researcher to meaningfully describe data with numerical indices or in graphic form.

Describe the process of using ethnographic research and define key terms related to this process.

Ethnographic research is particularly appropriate for behaviors that are best understood by observing them within their natural settings. The key techniques in all ethnographic studies are in-depth interviewing and highly detailed, almost continual, ongoing participant observation of a situation. A key strength of ethnographic research is that it provides the researcher with a much more comprehensive perspective than do other forms of educational research.

Key terms related to ethnographic research: thick description:

In ethnography, the provision of great detail on the basic data/information.

Threats to Internal Validity in Single-Subject Research

Independence of behaviors (are behaviors that are being measured dependent upon one another, or related?) Number of baselines (did an extraneous event cause the change during the introduction times?) Return to baseline level (level should quickly return if the intervention was the causal factor) Condition length (how long the baseline and intervention conditions are in effect) How many data points are gathered? Number of variables changed when moving from one condition to another (it is important that one variable be changed at a time, when moving from one condition to another) Degree and speed of change (magnitude with which the data change at the time the intervention condition is implemented) The baseline condition reveals the stability of the data

Discuss different types of observations and interviews. informal interview:

Less-structured forms of interview, usually conducted by qualitative researchers. They do not involve any specific type or sequence of questioning, but resemble more the give-and-take of a casual conversation.

Describe approaches to qualitative research. autobiography:

Life stories written by persons about themselves.

Identify the types of mixed-method design and discuss what types of studies benefit the most from using these different designs.

Mixed-methods research involves the use of both quantitative and qualitative research methods in a single study. The results of these separate methods are combined to present a more complete picture of the phenomenon under study than either method could produce on its own. The exploratory design: involves first conducting a qualitative study to discover important variables underlying a phenomenon and then conducting a quantitative study to discover relationships among the variables. This type of design is often used to develop rating scales in a new area of study. The explanatory design: involves first conducting a quantitative study and then conducting a qualitative study to expand upon the results of the quantitative study. The triangulation design: involves conducting both a qualitative study and a quantitative study (usually concurrently) and determining whether the results of the two studies converge on a single understanding of the underlying phenomenon. If the results do not converge, reasons for the lack of convergence need to be explored. Types of studies that benefit most: when you want to test something new A study looking at both quantitative and qualitative questions/methods In a study where some quantitative data can be explained better with qualitative data (ex. a negative relationship is found between students' amount of time spent on the computer and their grades [more time correlates with suffering grades]). In a study where quantitative data can help support the claims made from qualitative data.

Key terms related to ethnographic research: contextualization:

Placing information/data into a larger perspective, especially in ethnography.

A major threat to the internal validity of a causal-comparative study

Possibility of a subject selection bias. The chief procedures that a researcher can use to reduce this threat include matching subjects on a related variable, creating homogeneous subgroups, and using the technique of statistical matching. Other threats: Location instrumentation mortality LIM In addition, type 3 studies are subject to implementation, history, maturation, attitude of subjects, regression, and testing threats.

Key terms related to ethnographic research: member checking:

Procedure that involves asking participants in a qualitative study to check the accuracy of the research report.

Compare and contrast qualitative and quantitative research.

Qualitative studies that investigate the quality of relationships, activities, situations, or materials. The natural setting is a direct source of data, and the researcher is a key part of the instrumentation process in qualitative research. Quantitative: work is based on the belief that facts and feelings can be separated, that the world is a single reality made up of facts that can be discovered Aspects of both qualitative and quantitative research often are used together in a study. Increased attention is being given to such mixed-methods studies. Whether qualitative or quantitative research is the most appropriate boils down to what the researcher wants to find out.

What are graphs used for?

Researchers use various graphic techniques to summarize categorical data

Controlling Threats in a Single-Subject Study

Single subject designs are most effective in controlling for the following: Subject characteristics Mortality Testing History SMTH They are less effective with the following: Location Data collector characteristics Maturation Regression MRLD They are even weaker with the following: Data Collector bias Attitude Implementation DAI

Describe approaches to qualitative research. instrumental case study:

Study that focuses on a particular individual or situation with little effort to generalize.

Key terms related to ethnographic research: etic perspective:

The "outsider" or "objective" view of a culture's reality, especially in ethnography.

Key terms related to ethnographic research: holistic perspective:

The attempt to incorporate all aspects of a culture into an ethnographic interpretation.

Tell what differentiates a true experiment from a quasi-experiment. Also, be able to compare and contrast strong and weak experimental designs.

The essential ingredient of a true experiment is random assignment of subjects to treatment groups. Some true experiment designs include the randomized posttest-only control group design, the randomized pretest-posttest control group design, and the randomized Solomon four-group design. Quasi-experiements don't have random assignment of subjects. Some quasi-experiment designs include the matching-only design, the counterbalanced design, and the time-series design. Poor Experimental designs: Three weak designs that are occasionally used in experimental research are the oneshot case study design, the one-group pretest-posttest design, and the static-group comparison design. They are considered weak because they do not have built-in controls for threats to internal validity. In a one-shot case study, a single group is exposed to a treatment or event, and its effects are assessed. In the one-group pretest-posttest design, a single group is measured or observed both before and after exposure to a treatment. In the static-group comparison design, two intact groups receive different treatments. Strong Experimental designs: The randomized posttest-only control group design involves two groups formed by random assignment. The randomized pretest-posttest control group design differs from the randomized posttest-only control group only in the use of a pretest. The randomized Solomon four-group design involves random assignment of subjects to four groups, with two being pretested and two not.

Design a research proposal (What are the different parts of research-literature review, justification, procedures, instruments, participants, etc.).

The main body is the largest section of a proposal or a report and generally includes the problem to be investigated (including the statement of the problem or question, the research hypotheses and variables, and the definition of terms); the review of the literature; the procedures (including a description of the sample, the instruments to be used, the research design, and the procedures to be followed; an identification of threats to internal validity; a description and a justification of the statistical procedures used); and (in a proposal) a budget of expected costs.

What averages are used to summarize quantitative data?

The mean and median. The mean of a distribution is determined by adding up all of the scores and dividing this sum by the total number of scores. The median of a distribution marks the point above and below which half of the scores in the distribution lie. (The mode is the most frequent score in a distribution.)

Describe standard deviation

The most stable measure of variability; it takes into account each and every score in a distribution.

What is the normal distribution and what is a distribution curve?

The normal distribution is a theoretical distribution that is symmetrical and in which a large proportion of the scores are concentrated in the middle. A distribution curve is a smoothed-out frequency polygon. The distribution curve of a normal distribution is called a normal curve. It is bell shaped, and its mean, median, and mode are identical.

Key terms related to ethnographic research: multiple realities:

The recognition and acceptance of multiple views of reality, especially in ethnography.

Given the necessary information for a study, explain how to control for extraneous variables.

The researcher in an experimental study has an opportunity to exercise far more control than in most other forms of research. Some of the most common ways to control for the possibility of differential subject characteristics (in the various groups being compared) are randomization, holding certain variables constant, building the variable into the design, matching, using subjects as their own controls, and using analysis of the covariance. Building the variable into the design example: instead of holding the variable of "gender" constant by only including males in a study, purposefully including both males and females and analyzing the effects of both gender and the method on outcomes. When subjects are used as their own controls their performance under all treatments is compared. Analysis of covariance can be used to equate groups statistically on the basis of a pretest or other variables. The posttest scores of the subjects in each group are then adjusted accordingly. member checking: Procedure that involves asking participants in a qualitative study to check the accuracy of the research report. validity in qualitative research: Research in which the investigator attempts to clarify phenomena through carefully designed and controlled data collection and analysis.

Identify proper interviewing behavior as it relates to educational research.

The six types of questions asked by interviewers are background (or demographic) questions, knowledge questions, experience (or behavior) questions, opinion (or values) questions, feelings questions, and sensory questions. Respect for the individual being interviewed is a paramount expectation in any proper interview. The most effective characteristic of a good interviewer is a strong interest in people and in listening to what they have to say.

Key terms related to ethnographic research: culture:

The sum of a social group's observable patterns of behavior and/or their customs, beliefs, and knowledge.

Describe variability

The term variability, as used in research, refers to the extent to which the scores on a quantitative variable in a distribution are spread out. The most common measure of variability used in educational research is the standard deviation. The range, another measure of variability, represents the difference between the highest and lowest scores in a distribution.

Key terms related to ethnographic research: emic perspective:

The view of reality of a cultural "insider," especially in ethnography.

case study:

a detailed analysis of one or a few individuals.

biography:

a form of qualitative research in which the researcher works with the individual to clarify important life experiences.

Describe approaches to qualitative research. grounded theory study:

a researcher forms a theory inductively from the data collected as a part of the study.

action research:

a type of research by practitioners designed to help improve their practice.

threats to static-group

besides instrument decay testing maturation TIM all affect the static-group comparison

ethnographic research:

concentrates on documenting or portraying the everyday experiences of people, using observation and interviews.

Given a study, determine the best research design for the study and how to analyze the data.

experimental research: involves manipulating conditions and studying effects. correlational research: involves studying relationships among variables within a single group and frequently suggests the possibility of cause and effect. causal-comparative research: involves comparing known groups who have had different experiences to determine possible causes or consequences of group membership. survey research: involves describing the characteristics of a group by means of such instruments as interview questions, questionnaires, and tests. ethnographic research: concentrates on documenting or portraying the everyday experiences of people, using observation and interviews. action research: a type of research by practitioners designed to help improve their practice. case study: a detailed analysis of one or a few individuals. biography: a form of qualitative research in which the researcher works with the individual to clarify important life experiences. phenomenology: A form of qualitative research in which the researcher attempts to identify commonalities in the perceptions of several individuals regarding a particular phenomenon. (lived experiences!) grounded theory: A form of qualitative research that derives interpretations inductively from raw data with continual interplay between data and emerging interpretations. (try to create a theory grounded on data) Evaluation research: aims to improve the object or program being evaluated, usually by strengthening its delivery, implementation, and outcomes.

causal-comparative research:

involves comparing known groups who have had different experiences to determine possible causes or consequences of group membership.

survey research:

involves describing the characteristics of a group by means of such instruments as interview questions, questionnaires, and tests.

experimental research:

involves manipulating conditions and studying effects.

correlational research:

involves studying relationships among variables within a single group and frequently suggests the possibility of cause and effect.

Type of Score: A standard score

is a mathematically derived score having comparable meaning on different instruments.

Type of Score: A derived score

is a raw score that has been translated into a more useful score on some type of standardized basis to aid in interpretation. Age/grade equivalents are derived scores that indicate the typical age or grade associated with an individual raw score.

Type of Score: A raw score

is the initial score obtained when using an instrument

Type of Score: A percentile rank

is the percentage of a specific group scoring at or below a given raw score.

time-series is affected by

mortality instrument decay testing history attitude of subjects implementation MIT HAI

Develop a research study based on certain characteristics and data sets.

quantitative research: Experiments single subject correlational causal-comparative survey qualitative: ethnographic historical Mixed-methods Action research Differences between causal-comparative and correlational research: Causal-Comparative -- compares two or more groups of subjects -- involves at least one categorical variable (group membership) -- compares averages or uses crossbreak tables Correlational -- requires a score on each variable for each subject (single groups) -- investigates two or more quantitative variables -- analyzes data using scatterplots and/or correlation coefficients Both seek to explore relationships among variables. When relationships are identified through causal-comparative research (or in correlational research), they often are studied at a later time by means of experimental research.

Discuss different types of observations and interviews. observer roles:

ranging from complete participant, to participant-as-observer, to observer-as-participant, to complete observer. The degree of involvement of the observer in the observed situation diminishes accordingly for each of these roles.