Intro to research test 2

causal comparative research: def, how many groups, independent variable (ex post facto)

attempts to determine the cause or reason for existing differences -ex post facto: this research is done after the fact - Causal-comparative research begins with an effect and seeks possible causes. - Causal-comparative research includes two (or more) groups and one dependent variable. independent variable is not manipulated by the researcher. - The independent variable has occurred or is already formed. - The independent variable in causal-comparative studies is often referred to as the grouping variable. two categories: Retrospective casual-comparative research, Prospective casual-comparative research

Surveys are either sample surveys or census surveys

A sample survey is designed to sample participants and generalize to a population. A census survey samples every member of the population. - Census surveys are generally done with small and accessible populations.

survey research challenges

Participant response is a challenge, as a limited responses can skew data. It may be necessary to develop a survey instrument. Writing clear, unambiguous questions is a challenge.

Relationship Studies: Correlation coefficients

The Pearson r coefficient is the most common and most precise coefficient. Pearson r is used for continuous (ratio or interval) data. The Spearman rho coefficient is appropriate to use when at least one of the variables is rank or ordinal data. The phi coefficient is used when both variables are expressed as a categorical dichotomy (e.g., gender, political affiliation).

The Exploratory Sequential design

(QUALquan) • Collects qualitative data first in the sequence • More heavily weights qualitative data (QUAL) • Begins with a qualitative exploratory study (e.g., collecting observations, open-ended interviews) • Identify concepts and hypotheses to explore and test quantitatively (e.g., surveys, census, Likert-scale)

The Convergent Parallel

(QUAN + QUAL) • The strengths of qualitative data (e.g., context) offset weakness of quantitative data (e.g., ecological validity) • The strengths of quantitative data (e.g., generalizability) offset weaknesses of qualitative data (e.g., context dependence) • The most challenging mixed methods design

The Explanatory Sequential

(QUANqual) Design • Collects quantitative data first in the sequence • More heavily weights quantitative data (QUAN) • Quantitative findings determine type of data collected in second phase • Use the qualitative analysis to explain or elaborate the quantitative results • e.g., survey followed by focus groups to explain findings

External validity threats can be divided into two categories:

- 'Generalizing to whom' threats Threats affecting groups to which the study can be generalized - 'Generalizing to what' threats Threats affecting the settings, conditions, variables, and contexts to which the results can be generalized

Experimental studies in educational settings often have two problems:

- A lack of sufficient exposure to treatments (i.e., treatments are too short or diffuse). - Failure to make treatments significantly different from one another (e.g., an experimental instructional program in math may not be different enough from the comparison math instructional program).

Types of participant observation in Ethnographic Research (3)

- Active participant observer e.g., Researchers often negotiate roles as teachers' aides - Privileged, active observer e.g., Researcher may serve as an aide but is able to step back and forth within the role providing opportunities for observation - Passive observer e.g., Researcher only observes, no active role

narrative research forms (6)

- Autobiographies - Life writing - Personal narratives - Life histories - Narrative interviews - Oral histories

Case Study Research: Definition and Purpose

- Case study research is an all-encompassing method that includes specific design, data collection, and data analysis. - Case study research yields concrete knowledge that readers personally relate to their own experiences. - Case study research is a qualitative approach. • Case study research is appropriate when the researcher wants to answer: - a descriptive question (e.g., what happened?) or - an explanatory question (e.g., how did something happen). -what was the process.

Cohort Survey

- Cohort surveys examine one population selected at a particular time period but multiple samples taken and surveyed at different points in time. e.g., A researcher could study senior, preservice teachers attitudes towards technology in the classroom. The researcher might administer the survey in 2018 to a sample of senior-level, preservice teachers. PICKING 200 RANDOMLY OUT OF THE 1200 Then, the researcher would return to the same population and randomly select a new sample to survey (e.g., in 2019, 2020, 2021). PICKING A NEW 200 RANDOMLY OUT OF THE SAME 1200 different individuals per survey BUT THE SAME GROUP PF PEOPE

Correlational Research: def and purpose

- Correlational research involves collecting data to determine whether and to what degree a relationship exists between two or more quantifiable variables. - Correlational studies often examine numerous variables believed to be related to a major, complex variable (e.g., achievement). - The purpose of correlational research is to determine relations among variables or to use these relations to make predictions. many variables and how they relate

Wolcott (1999) identified three conceptual areas that help to focus ethnographic researchers as they address culture:

- Cultural orientation Physical space where group resides and activities in which the group engages - Cultural know-how Daily activities of a group - Cultural beliefs Why group does what it does

Longitudinal Surveys

- Data are collected at two or more time points. - One of the challenges when conducting longitudinal studies is attrition. Time can also be another obstacle - Four types: Trend survey Cohort survey Panel survey Follow-up survey

Experimental Research:Definition and purpose

- Experimental research is the only type of research that can test hypotheses to establish cause-effect relations. -one causes the other (the strongest evidence for cause and effect after replicating the study) -Experimental research is the most structured of all research. - Experimental studies often examine comparisons between or among groups.

Types of Mixed Methods Research Designs: Three basic mixed methods designs

- Explanatory sequential (QUANqual) - Exploratory sequential design (QUALquan) - Convergent parallel design (QUAN + QUAL)

Follow-up Survey

- Follow-up surveys address development or change in a previously studied population, some time after the original survey. e.g., A researcher studies senior, preservice teachers attitudes towards technology in the classroom and administered the survey in 2012. A number of years later, in 2015, a researcher identifies individuals from the original study and surveys them to examine changes in attitudes. -this won't be in your original plan it many just become needed later on in study

Guidelines for participant observation

- Gaining entry and maintaining rapport - Reciprocity - Tolerance for ambiguity Fieldwork often does not proceed at the speed and intensity desired. Ethnographers must be patient. - Personal determination coupled with faith in oneself Self-doubtisexpected. Keepitincheck. - Letting go of control

Reactive arrangements (4)

- Hawthorne effect: Any situation in which participants' behavior is affected because they are in a study. - John Henry effect:(Compensatory rivalry): Members of the control group compete with the experimental group. Members of control group feel threatened or challenged by being in competition and preform beyond "normal" - Placebo effect: To combat compensatory rivalry, researchers attempt to give control groups a placebo, not the experimental treatment, but something to decrease the perception that they are in the control group. Participants should perceive they are all getting the same thing. - Novelty effect: When participants are engaged in something different this may increase attention, interest, behavior, learning, etc., just because it is something new.

experimental group vs control group vs comparison

- In experimental research studies the group that receives the treatment is the experimental group. - The group that does not receive the treatment is called the control group. Sometimes groups are comparison groups that receive alternative treatments (e.g., two types of instruction in a content area).

Experimental Research: Manipulation and control

- In experimental studies, researchers directly manipulate independent variables and control, or remove, the influence of extraneous variables. - It is challenging to control all the relevant extraneous variables. Participant variables - Organismic (e.g., age) Environmental variables (e.g., school or teacher effects) Concentrate on controlling variables that affect or interact with the dependent variable.

Prediction Studies: Data collection (attrition)

- In prediction studies all measures should be valid measures. It is especially critical that the criterion variable be validly measured. - In prediction studies, the predictor variables are generally administered prior to the criterion variable (e.g., SAT and university GPA). Attrition is a problem in some prediction studies: the action or process of gradually reducing the strength or effectiveness of someone or something through sustained attack or pressure.

Relationship Studies: Data analysis and interpretation

- In relationship studies, scores from one variable are correlated with scores for another variable, or scores for several variables are correlated with a particular variable of interest. - The result is a number of correlation coefficients. - The method of calculating a correlation coefficient depends upon the nature of the data (nominal, ordinal, interval, ratio)

Narrative Research: Definition and purpose

- Narrative research is the study of how different humans experience the world around them. - Narrative researchers collect data about people's lives and collaboratively construct a narrative about the experiences and meanings they contribute to the experiences. - Narrative research has often been conducted in numerous disciplines (e.g., history, anthropology). -The narrative research process is highly personal and requires care and sensitivity.

Examples of variables investigated in causal-comparative studies: (5)

- Organismic variables (e.g., age, ethnicity, sex) - Ability variables (e.g., achievement) - Personality variables (e.g., self-concept) - Family-related variables (e.g., SES) - School-related variables (e.g., type of school, size of school)

Panel Survey

- Panel surveys involve a sample in which the same individuals are studied over time. e.g., A researcher studies senior, preservice teachers attitudes towards technology in the classroom over a three year period. The researcher might administer the survey in 2019 to a sample of senior-level, preservice teachers. Then, the researcher would survey the same individuals each of the three years (i.e., in 2020 and 2021). same individuals per survey

Relationship Studies: Data collection

- Researchers first identify variables to be correlated. - Variables should be purposely identified. - A smaller number of carefully identified variables is preferable to a larger number (e.g., shotgun approach). - After identifying variables the researcher next identifies the appropriate population and sampling procedure to select participants for the study. - Data may be collected all at one session or at several sessions conducted in close succession.

Correlational Research Process: Participant and instrument selection

- Sample must include at least 30 participants. Sample size effects the strength of a correlation coefficient. - Instruments must reflect intended variables. - Select instruments reliable and valid for the intended purpose of the study.

Correlational Research Process: Design and procedure

- Scores for two or more variables of interest are obtained for each member of the sample. - Pairs of scores are then correlated (compared to each other to see if one means the other is true) (e.g., self-concept and achievement).

Statistical Significance

- Statistical significance is the likelihood that a relationship between two or more variables is caused by something other than chance. - Test provides a p-value, representing the probability that random chance could explain the result. - The statistical significance is set by the researcher, rule of thumb or convention dictates 5% or 0.05. Result would occur by chance no more that 5 times out of 100. meaning the correlation works 95% of the time

Survey Research Definition and purpose

- Survey research involves collecting data to test hypotheses or to answer questions about people's opinions on some problem or issue. - A survey is an instrument to collect data that describe one or more characteristics of a specific population.

The Causal-Comparative Research Process: Design and Procedures

- The basic causal-comparative design involves selecting two groups that differ on a variable of interest and comparing them on a dependent variable. - Grouping variables must be operationally defined (e.g., training versus no training). - Determine the equality of groups. Homogeneity mitigates alternative explanations of the research findings.

Experimental Research: The experimental process

- The steps in the experimental research process are the same as in other types of research. 1. Selectinganddefiningaproblem 2. Selectingparticipantsandmeasuringinstruments 3. Preparingaresearchplan 4. Executingprocedures 5. Analyzingthedata 6. Formulatingconclusions

There are several mechanisms to assist readers in identifying a mixed methods study.

- The title includes terms. e.g., mixed methods, convergent parallel, simultaneous - The purpose statement or the research questions indicate mixed methods were used. - The researcher states that both methods are used in the study.

The Causal-Comparative Research Process:Control Procedures (3)

- Three procedures attempt to control for extraneous variables. Matching: Researchers can attempt to equate groups on one or more variables. -For example, a researcher comparing two types of instruction might control for prior achievement. To do this, the researcher would match each participant with another with the same or similar achievement. Comparing homogeneous groups or subgroups:Researchers can also compare groups that are homogeneous with respect to an extraneous variable. -For example, the researcher may select only high- achieving students for his study. Analysis of Covariance (ANCOVA):Researchers can use this statistical technique to adjust for initial differences on variables used. - For example, the researcher could measure prior knowledge and use those scores as a covariate. When we gave the two groups a test of math abil-ity prior to introducing the new teaching methods, we found that the group to be taught by Method Y scored much higher than the group to be taught by Method X. This difference suggests that the Method Y group will be superior to the Method X group at the end of the study just because members of the group began with higher math ability than mem-bers of the other group. Analysis of covariance sta-tistically adjusts the scores of the Method Y group to remove the initial advantage so that, at the end of the study, the results can be fairly compared, as if the two groups started equally.

Trend Survey

- Trend surveys examine changes over time in a particular population defined by a particular trait or traits. e.g., A researcher could study senior, preservice teachers' attitudes towards technology in the classroom. The researcher might administer the survey annually to a sample of senior-level, preservice teachers. DIFFERENT PEOPLE EVERY TIME

Narrative Research Process: data collection methods

- We would conduct interviews and would like to observe our participant's classroom. - We would also likely examine other classroom artifacts.

Narrative research can be categorized based upon five characteristics:

- Who authored the account - Scope of the narrative - Who provides the story - Theoretical/conceptual framework - Whether the elements are included in one narrative

Distributing the questionnaire

-Mail: cheap but small response rate -Email: fast and can receive multiple replies but not everyone has email and they could do the survey multiple time for the reward -Telephone: High response rate but not in depth data and requires phone list -Personal administration: efficient but time consuming -Interview: high return rate but no anonymity and time consuming -web based: tools support design and analysis of survey instruments. (can provide templates and pilot

good response rate

50%

Problems to Consider in Interpreting Correlation Coefficients

Ask the following questions: - Was the proper correlation method used? - Do the variables have high reliabilities? Low reliabilities lower the chance of finding significant relations. - Is the validity of the variables strong? Invalid variables produce meaningless results.

Conducting and Analyzing Multiple Case Studies (7)

Cross-site analysis strategies - Unordered meta-matrix Master charts with descriptive data from each site - Site-ordered descriptive matrix Ordered meta-matrix orders sites based upon critical variable of interest - Site-ordered predictor-outcome matrix Includes more explanation and interpretation Order sites to understand variables that appear to contribute most directly to outcomes - Time-ordered meta-matrix Uses chronology as an organizing variable for a meta-matrix - Scatterplots Visual displays of data from all the sites - Site-ordered effects matrix Presents potential cause-effect relations Presents manifest functions, latent functions, unintended consequences - Causal models Extends case study analysis Presents possible cause-effect relations among variables

The Causal-Comparative Research Process :Basic causal-comparative designs

One group is exposed to an independent variable while the other group is not. Both groups are measured on a dependent variable Two groups are exposed to different independent conditions. Both groups are then measured on a dependent variable.

Culture:

Set of attitudes, values, concepts, beliefs, and practices shared by members of a group.

story

The story is the outcome of the narrative research.

- Correlational studies are designed to test hypotheses regarding expected relations based on:

Theory Experience

mode, mean, median (which one is the the most relevant for causal comparative research)

mode: frequency mean: Average median: middle mean because it tells is the most about the data

Retrospective casual-comparative research

start with effects and investigate causes. - More common in educational research. e.g., A researcher interested in the benefits of an exercise program on reducing stress may select a group of people who had enrolled in a stress- reduction exercise class and those who had not and compares their stress levels.

Prospective casual-comparative research

start with the causes and investigate the effects. - What is the effect of X? A researcher may hypothesize that those children that attend dance classes during elementary school have higher self-esteem when in middle school. She would identify a group of middle-school children who had dance classes in elementary school and a group of those who did not and compare their self- esteem.

how to know when you have enough information

you start hearing the same thing over and over again

Correlational Research: correlation coefficient (weak, moderate, strong)

• A correlation coefficient indicates the size and direction of the relationship between two variables • Correlation coefficients range from -1 to 1. • A correlation of 0 indicates no relationship. • A correlation near 1 or -1 indicates a large effect size. • A positive or negative represents the direction of the relation. - Correlation coefficients between +.35 and -.35 represent a weak relationship or no relationship. - Correlation coefficients between +.35 and +.65 or between -.35 and -.65 represent moderate relationships. - Correlation coefficients between .65 and 1.0 or between -.65 and -1.0 represent strong relationships. - Correlation coefficients do not represent percentage of relation between variables. The square of the correlation coefficient indicates the amount of common variance (shared variance). - e.g., a correlation of .50 indicates 25% common variance. Common variance indicates the extent to which variables vary in a systematic way.

Cross-sectional surveys

• Data are collected from selected participants at a single point in time. • These studies provide a snapshot of current behaviors, attitudes, and beliefs. • These survey studies are not intended to provide insight into trends or changes over time. • E.g., A study that examines the self-concept of all pre- service teachers at a given university this semester.

Digital tools for use with narrative research

• Dragon Dictation (Free Mobile App) - Instantly changes spoken word into text that can be emailed or edited. - Can record observations, field notes, interviews, reflections. - Five times faster than typing

Ethnographic Research: Definition and Purpose and prosses

• Ethnography is the study of cultural patterns and perspectives of participants in their natural settings. • Ethnographers engage in long-term study of particular phenomena to situate understandings in a meaningful context. • It is carried out in a natural setting, not a laboratory. • At first the ethnographer eases into the setting and refrains from asking participants questions. • It cannot provide an exhaustive, absolute description of anything.

Threats to Internal Validity (7)

• History - Any event occurring during a study that is not part of the experimental treatment but that may affect the dependent variable represents a history threat. - Longer-lasting studies are more prone to history threats. • Maturation - Maturation refers to physical, intellectual, and emotional changes that naturally occur within participants over a period of time. • Testing - Testing as a threat to internal validity is demonstrated when taking a pretest alters the result of a posttest. • Instrumentation - Instrumentation is a threat to internal validity when the instrumentation is either unreliable or is changed between pre- and post-testing. • Statistical regression - Extremely high or low scores tend to regress to the mean on retesting. • Differential selection of participants - Participants in the control and experimental groups differ in ways that influence the dependent measure. • Mortality - Mortality refers to attrition or a reduction in the number of research participants over time as individuals drop out of a study. Do not assume attrition is random.

Threats to Experimental Validity (internal, external, construct)

• Internal validity - Internal validity refers to the degree to which observed differences in the dependent variable are a direct result of manipulation of the independent variable and not some other variable. - Internal validity is concerned with rival explanations for an effect. • Construct validity - Construct validity refers to inferences about the variables or constructs in a study. - The degree to which a variable represents what it claims, or purports to be. • External validity - External validity, also called ecological validity, refers to the degree to which the results from a study are generalizable to other groups. - When researchers increase the internal validity of their study, they decrease their external validity. this is because the generalizable population does not have all that control. despite this it is better to air on the side of control on internal validity

Mixed Methods Research: Definition and Purpose

• Mixed methods research designs involve the collection, analysis, and "mixing" of quantitative and qualitative research designs.

Case studies can be described as: (3) pdh

• Particularistic: Focused on a particular phenomenon, situation, or event • Descriptive: Includes a thick description of the phenomenon under study • Heuristic: illuminate understanding of the phenomenon beyond original knowledge

Threats to External Validity (7)

• Pretest-treatment interaction - This threat occurs when participants respond differently to a treatment because they have been exposed to a pretest. At times, unobtrusive measures can be used as pretests, to limit this threat to validity (using something they haven't seen before) • Multiple-treatment interference - This threat occurs when previous treatments cross- over into a current experiment. This makes it challenging to determine the effectiveness of the later treatment. • Selection-treatment interaction - When a study's findings only apply to the groups selected and are not representative of other groups. • Specificity of variables - When researchers do not adequately define their variables, instruments, or population, it makes it difficult to determine how well the findings will generalize to an alternative population. • Treatment diffusion - When different treatment groups communicate with and learn from each other, the treatments are no longer distinctly different. The treatments overlap. one treatment diffuses to the other • Experimenter effects - Experimenter effects occur when characteristics or behaviors of the experimenter influence the participants' responses. (experimenter bias) • Reactive arrangements - These threats are also referred to as participant effects. These threats are associated with differences in participants' behavior, feelings, and attitudes because they are in a study.

questionnaires vs interviews

• Questionnaires are a written collection of survey questions answered by a selected group of participants. most often used - Usually mailed or emailed - New technologies for on-line surveys (e.g., SurveyMonkey.com, Zoomerang.com, etc.) • Interviews are oral, in-person question and answer sessions between the researcher and a respondent.

Types of Ethnographic Research (3)

• Realist ethnography: Objective, scientifically written • Critical ethnography: Written to advocate against inequalities and oppression • Ethnographic case study: A case analysis of a person, event, activity, or process set within a cultural perspective

Conducting a Questionnaire Study (5) (unstructured vs structured)

• Stating the problem - Participants are more likely to respond to questionnaires that they perceive as significant. - Researchers should set objectives for the type of information needed from a questionnaire. • Constructing the questionnaire - Developing a questionnaire requires skill and time. - Researchers should plan content and format of the questionnaire. - Most surveys include one of two types of items. Structured items require participants to choose among provided responses (e.g., check a response box). Unstructured items have the respondent has complete freedom of response (e.g., fill-in answer). Guidelines to consider when constructing a questionnaire: - Include only items that relate to the objectives of the study. - Collect demographic information about the sample if you plan to make comparisons between different. - Focus each question on a single concept. - Define or explain ambiguous terms. - Include a point of reference to guide respondents in answering questions. - Use examples if the item format is unusual. - Leave sufficient space for respondents to write their responses to open-ended items. - Group items and response options together. - Write directions for the respondents. - Include the purpose of the study at the top of the instrument. - Pilot test the questionnaire. • Pilot testing the questionnaire - Pilot testing provides information about deficiencies and suggestions for improvement. - Choose individuals who are thoughtful, critical, and similar to intended research participants. - Ask the pilot-test group to make comments and suggestions concerning directions, recording procedures, and specific items. - Reviewers should also note questions omitted. - Incorporate feedback from your pilot administration and make any appropriate changes. • Preparing the cover letter - Cover letters/email should accompany all surveys. - Cover letters/email should be brief. - Cover letters/email should be addressed to a specific person. - The cover letter should explain the purpose of the study. - The cover letter/email should include a statement regarding the importance and significance of the study. - The cover letter/email should include how results from the study will be shared with respondents and interested others. - If possible, the letter/email should state a commitment to share the results of the study when included. - The cover letter/email should share the researcher's address, phone number, or email address. - The cover letter should include endorsements from organization, institution, group, or administrator that the respondent is likely to know or recognize to add credibility to your study. Endorsers can cosign the cover letter - Pilot test the cover letter.

Conducting Follow-Up Activities

• The higher the return rate the improved confidence you will have in your study. • Follow-up strategies include: - Send out a reminder postcard/email to increase response rates. - Send out a second new cover letter/email and questionnaire to increase response rates. - Consider phone-call reminders to increase response rates.

Relationship Studies

• The purpose of relationship studies is to gain insight into variables or factors that are related to a complex variable (e.g., retention, academic achievement). • Relationship studies help researchers to determine which variables may be suitable for future research. • Relationship studies provide insight into which variables should be controlled for in future causal-comparative or experimental studies. one complex variable and many related factors

Narrative Research Process: Restorying

• The restorying process 1.Researcher conducts the interview and transcribes the audiotape. 2.Researcher transcribes the data based upon key elements that are identified in the story. 3.Researcher organizes the story into a chronological sequence with attention to setting, characters, actions, problems, and resolutions. The researcher then invites the participant to collaborate on the final narrative.

Tabulating Questionnaire Responses

• Use scannable answer sheets if possible. - Accuracy • Code open-ended answers. - Consider available qualitative software analysis packages. • Report total sample size. • Report the overall percentage of return and response rates per item. - e.g., Percentage that answered 'yes' and percentage that answered 'no' • organize by mean scores by cluster or area of a survey, such as comparisons by participant characteristics. - e.g., Percentage of men or women with a particular response. • Reports of survey studies will vary by the underlying purpose of the survey.

Prediction Studies

• When two variables are highly related, scores on one variable can be used to predict scores on the other variable. - The variable used to predict is called the predictor. - The variable that is predicted is called the criterion. • A prediction study is used to determine which variables are the most highly related to the criterion variable. - More than one variable can be used to make predictions. finds the relationship between the predator and the complex variable