Research

Theoretically, the more control the researchers have over the experimental conditions, __________________. a) the lower the internal validity b) the lower the internal reliability c) the higher the internal reliability d) the higher the internal validity

Higher the internal validity

Events occurring during the time of the study that are unrelated to the treatment or independent variables are known as ______. a) instrumentation b) statistical regression c) history d) maturation

History

Threats to internal validity

History Maturation Testing Instrumentation Diffusion of treatment Regression towards the mean Selection bias Attrition Combination of selection & other threats diffusion or imitation of treatment special treatments

Scientific Method Steps

Identify the problem, Develop research, identify and develop research design, develop and identify techniques/instruments, collect & analyze data, draw conclusions, report the data

External Validity: Multiple-Treatment Interference

In some studies, participants are exposed to multiple treatment conditions• Makes it difficult to draw conclusions about a given treatment when it is evaluated in the context of other treatments

Quantitative Research Designs

Research that is based on measurement and quantification of data (i.e., all data turned into numbers)

Descriptive Designs 4 types:

Survey: involves the use of self-report to clarify the perceptions, attitudes, or behaviors of a target group• There is no IV• Data is collected solely to describe some aspect of a particular group• Observational: involves observing the behavior of a target group• This design still clarifies personal characteristics , but avoids problems of self-report• Correlational: used to identify and understand the relationship between multiple variables or how several variables of interest covary• Cannot draw any conclusions regarding causation• For some great examples, go here: Causal-comparative: used to identify effects of an IV after the fact• The IV is outside the researcher's control• The researcher compares two or more groups who had different experiences (experiences = the IV)• This is a quasi-experimental design

Construct Validity

The conceptual basis (construct) underlying the effect

external validity: test sensitization:

The effect of a previous test on subsequent performance • Purpose of a pretest is to measure the participant's standing on a particular variable before that person receives the experimental manipulation or treatment • Pretest sensitization: the effect of a pretest on a person's response to an intervention • Results may not generalize to people who did not have the pretest

Testing

The effects that taking a test once can have on subsequent performance•

Statistical Conclusion Validity [COVERED WITH RESULTS]

The extent to which a relation is shown and the extent to which the experiment detects effects if they exist

External Validity: sample characteristics

The extent to which characteristics of the sample represent the target population

Internal validity

The extent to which the intervention can be considered to account for the results (as opposed to a confounding variable) • Relates to the amount of control researchers had over the study

External validity

The extent to which the results can be generalized to circumstances other than those in the particular experiment• Relates to how true to life (generalizable) the study is•

Statistical Conclusion validity

The extent to which we can be certain that the researcher has drawn accurate conclusions about the statistical significance of the research.

When writing the reflection section of your annotation, it is important to focus on what is relevant to your study and how you plan to integrate the information into your work. True or false

True

2 Ethical Theories (Beauchamp & Childress)

Utilitarianism: the end justifies the means •Deontological: the outcome is less important than following a rule or principle

In the discussion section, the researchers should summarize the results by ______. a) not using any numbers or statistics b) relating results to previous research c) relating results to theories and practiced) all of the above d) all of the above

a) not using any numbers or statistics b) relating results to previous research c) relating results to theories and practice d) all of the above

The general purpose of a literature review section is ______. a) provide an argument for conducting the ) selecting variable c) rule out particular sample population d) rule out characteristics of population

a) provide an argument for conducting the research

Maturation

• Changes over time that result from physical/psychological processes within participants

Variables:

• Dependent variable• Changes as a result of changes in independent variables: Dependent variables depend on independent variables Extraneous variable: Uncontrolled and/or unknown variables that can impact the dependent variable, Not controlled by the researcher Control Variables: An extraneous variable that the researcher has identified and addressed in the method• Three methods for controlling for extraneous variables• Build the variable into the design and control for its effects (make it an independent variable)• Remove possible effects of the extraneous variable (sample from one level of the variable)• Control through statistical methods after the study has been conducted (e.g., use an ANCOVA)

external validity: History & Treatment Interaction

• Events occurring at time of treatment/intervention may affect outcome• Hard to control this, but can address by extending study beyond impact of uncontrolled event

Attrition

• Loss of participants • Problem in longitudinal studies • Direct function of time • Most people drop out early• May lead to systematic differences between the groups• Differential attrition across groups • Participants more likely to remain in study if they are doing something interesting, something that has little or no cost or adverse side effects, seems plausible, and is effective

External validity: Reactivity of assessment

• Most psychological assessments are obtrusive • If awareness of assessment leads people to respond differently, we call that reactive • Need to determine whether the changes carry over to people's actual experiences in their everyday lives

Threats to external validity

• Sample Characteristics • Interaction Between Sample and Treatment • Stimulus Characteristics and Settings• Reactivity of Experimental Arrangements • Multiple-Treatment Interference • Novelty Effects • Reactivity of Assessment • Test Sensitization • Timing of Measurement•

Selection Biases

• Systematic differences between groups before any experimental manipulations or interventions

When the experimenter communicates to the participants, most often subtly, what outcomes he or she would like to achieve, this is known as ________. a) the Hawthorne effect b) the disruption effect c) the experimenter effect d) the description effect

experimenter effect

Thorndike and Thorndike-Christ noted that many tests and measures are designed to "reflect the values and experiences of typical ______." a) upper class b) lower class c) poor/working class d) middle class

middle class

Descriptive Designs

study designs in which the researcher defines a problem and variable of interest but makes no prediction and does not control or manipulate anything

Two different types of populations include ______. a) accessible population and intended population b) target population and accessible population c) target population and observable population d) observable population and accessible population

target population and accessible population

Theoretically, the more control the researchers have over the experimental condition, ______. a) the lower the internal validity b) the higher the internal reliability c) the lower the internal reliability d) the higher the internal validity

the higher the internal validity

`The purpose statement is typically found in the abstract review section of an article, toward the end of the review. True or False

False

Experimental Precision

*reliability* ability of an instrument to read consistently, w/in a narrow range

Scientific Realism

- assumes the world is composed of layers of causal structures; some causal structures are easy to observe and others are not; researcher's job is to identify causal structures and how they interact to produce an effect

3 General Categories of Research Question & Hypothesis

1) descriptive 2) difference 3) relationship

Types of Knowledge Approaches

1. Description:Consists of attempts to describe natural, social, or psychological events 2. Prediction:Involves developing ways to predict identified outcomes 3. Improvement: Involves developing information designed to determine the effectiveness of interventions. 4. Explanation: Researchers frame questions and problems in terms of theories or explanations of phenomena

Steps in an evaluation

1. Formation of evaluation team 2. Identification of relevant stakeholders 3. Determination of a focus for the evaluation 4. Identification of evaluation model & methods 5. Selection of evaluation methods and designs 6. Selection of measures for the evaluation 7. Collection of the data 8. Analysis of the data

Ways to determine reliability

1. Test-Retest 2. Alternate or Parallel Form 3. Internal Consistency 4. Inter-Rater

Qualitative Designs 5 qualities

A naturalistic approach:Involves the collection of data in the natural environment (as opposed to in the lab) The use of descriptive data:Data is collected and presented through language and pictures§An emphasis on process§Focus on the way things are done rather than the outcomes/accomplishments§ An inductive approach:Researchers explore what comes up, as opposed to the deductive approach, in which researchers hypothesize what is there and look for support for the hypothesis A focus on meaning:Understanding the meaning that certain things in the environment have for the people in that environment

Quasi Experimental Design

An experiment that does not require random assignment to conditions. (pre-test, one group pretest)

History

Any event during the time of the study other than the independent variable that could account for the results• Such an event could occur in the experiment or outside of the experiment

General Principles of Ethics

Autonomy, Beneficence, Justice, Nonmalaficence, Fidelity

Quality Research Involves

Can be replicated• Is generalizable to other settings • Is based on a reasonable rationale and linked to a theory or theories • Is not based on political beliefs • Is objective* (though still subject to human error)• Common errors researchers make: • Inaccurate observation • Overgeneralization • Selective observation•

Instrumentation

Changes in the measuring instrument or measuring procedures over time • May also include response shift • Changes in a person's internal standard of measurement • Reflects a change in values, perspective, or criteria that lead to evaluation of the same or similar situations, behaviors, or states in a different way

When the purpose statement denotes an attempt to identify phenomena, events, or exercises, he or she is using ______. a) clarifying methods b) quantitative methods c) ethnographic methods d) unreliable method

Clarifying methods

Threats to construct validity impact the _______ that can be drawn from the findings. a) concepts b) conclusions c) controls d) cues

Conclusions

Construct validity

Construct validity addresses whether the method of measurement actually measured the construct of interest • Looks at the conceptual basis (construct) underlying the effect (What IS the intervention?) • The findings are what they are; threats to construct validity impact the conclusions that can be drawn (Why did this intervention lead to change?)

Researchers use ______ to summarize the data collected based on the sample in a particular research study, which includes calculating the measures of central tendency. a) inferential statistics b) descriptive statistics c) nonparametric statistics d) regression statistics

Descriptive statistics

Writing the method

Design, participants, measures, procedures

___________ type threat to internal validity occurs when the experimental and control groups are selected based on different criteria or when participants are assigned to groups differentially and not by random assignment. a) attrition b) differential selection c) maturation d) history

Differential selection

Qualitative

Direct observation •Interview •Triangulation methods •Documents •Audiovisual materials

Quantitative

Direct observation and behavioral measures •Self-report and inventories •Ratings of others' behavior •Physiological approaches •Interviews

The three types of hypotheses mentioned by Houser arenull, nondirectional, and ______ a) working b) bidirectional c) major d) directional.

Directional

six steps in using a case study approach

Establishing the boundaries of the case §Identifying themes of emphasis§ Focusing on specific patterns of data§ The use of triangulation in data interpretation§ Considering alternative views§ Determining the appropriate generalizations from the case

Annotations should be an exact copy of the abstract. True or False

False

One of the early uses of virtual reality in counseling has been with the treatment of individuals who have been diagnosed with ______. a) borderline personality disorder b) schizophrenia c) PTSD d) insomnia

PTSD

Research vs Program Evaluation

Purpose: Research: test theories, develop practive Evaluation: Decision making Audience: Research: Professional and scientifically community Evaluation specific group or communities such as funding sources or governmental agencies Method: Research: True experimental, quasi experimental, nonexperimental, qualitative Evaluation: ^^

Sampling Methods used in Qualitative Research

Purposive sampling maximal variation sampling typical sampling snowball sampling quota samping

External validity: Novelty effects

The possibility that the effects of an intervention may in part depend on their innovativeness or novelty in the situation• Novelty effects wear off over time, but are hard to evaluate

External validity: timing of measurement

The results of an experiment may depend on the point in time that assessment devices are administered• Will the same results occur if the experiment is administered at a different time?

Statistical Regression

The tendency for extreme scores on any measure to revert to the mean of a distribution when the measure is readministered

It is important to note any biases in the literature review. True or False?

True

In the discussion section, researchers may need to propose ______. a) concrete links to the data b) alternative explanations when the results do not meet expected outcomes or are not consistent with the theories addressed c) the type of statistics the reader should use d) to what extent the instruments were standardized

alternative explanations when the results do not meet expected outcomes or are not consistent with the theories addressed

______ concerns the consistency of a measure based on content, for instance, when two forms of the measure are administered to a sample and reliability coefficients are obtained between them. a) scorer reliability b) alternate-form reliability c) alternate-form validity d) test-retest reliability

alternative form reliability

The researchers' acknowledgement of possible limitations demonstrates ______. a) honesty b) integrity c) an understanding of the cautions that should be taken when interpreting results d) weakness

an understanding of the cautions that should be taken when interpreting results

Which of the following statement(s) is/are NOT true when evaluating articles in the professional literature? a) any error or limitation of a study should cause you to totally reject the findings b) a general purpose of the literature section is to provide an argument for conducting the study c) it is important to note any biases in the literature review d) there is need for the researcher to define new concepts in the literature review

any error or limitation of a study should cause you to totally reject the findings

Post-positivism

assumes individual perceptions of social environment and an event influence how they behave (including researchers, who try to "bracket" and remove biases); also assumes we can only approximate the "truth" with research•

Constructivism

assumes there is no true reality, but multiple, socially-constructed realities; research is about accessing participants' "lived experience"; values of researchers cannot be removed from the research process•

The purpose statement should not be based on the argument developed in the literature review. True False

b) false

In evaluating the discussion section, it is necessary to review the potential threats to validity based on ______. a) the research design used b) the methods of selecting participants c) the tactics used for coding d) both the research design used and the methods of selecting participants

both the research design used and the methods of selecting participants

Qualitative Designs

case study, ethnography, phenomenology, grounded theory, Histroical

A common sampling approach used by researchers because many of the methods are more difficult to implement is known as ______. a) convenience sampling b) simple random sampling c) cluster sampling d) equal size sampling

convenience sampling

______ scores are based on a predetermined level of performance to a comparison study.

criterion-referenced

Random assignment to groups _________ the effects of some threats to validity. a) increases b) decreases c) escalates d) doesn't change

decreases

____________ validity refers to the extent to which the researchers can replicate the actual events occurring in the natural environment and thus can generalize the results back to the population. a) internal b) external c) replication d) extraneous

external

In the discussion section, it is important that the researchers address ______. a) how the independent variable was chosen b) how the dependent variable was chosen c) how the research questions or hypotheses relate to the findings d) what themes were used in the study

how the research questions or hypotheses relate to the findings

Random assignment to groups ______ the validity of a study. a) increases b) eliminates c) decreases d) doesn't change

increases

Which types of statistical methods are used by researchers to generalize results back to the population of interest? a) qualitative statistics b) parametric statistics c) descriptive statistics d) inferential statistics

inferential statistics

Which threat to external ecological validity refers to how events occurring at the time of the intervention or treatment affect the outcome? a) measurement of the dependent variable b) interaction between time of measurement and treatment effects c) interaction between history and treatment effects d) novelty or disruption effects

interaction between history and treatment effects

Which type of validity do researchers generally agree is of higher priority when considering the trade-offs between the different types of validity? a) internal b) external c) construct d) statistical conclusion

internal

Statistics is commonly defined as the science of conducting studies to organize, summarize, and ______ information. a) set up b) infer c) interpret d) understand

interpret

As consumers of research, practitioners are particularly interested in the discussion because ______. a) it provides an overview of the participants b) it provides an overview of instrumentation c) it delineates the variables used d) it provides what worked and how it may be applied

it provides what worked and how it may be applied

The extent to which the results of an experiment can be generalized from the sample that participated in it to a larger group of individuals, or population is known as ______. a) population validity b) sample reliability c) population reliability and validity d) population reliability

population validty

In the discussion section, the researchers will want to relate the results to ______. a) format utilized by the type of data collected b) format which delineates the type of statistical methods used c) manner which explains the tactics or strategies for coding used d) previous research that either supports or refutes prior findings

previous research that either supports or refutes prior findings

__________ is about how we invite potential participants from the population of interest to participate in a study, while __________ is about how we assign participants to groups once they agree to participate. a) random selection; random assignment b) random assignment; random selection c) external validity; internal validity d) internal validity; external validity

random selection, random assignment

The literature review should include ______ a) only supportive evidence of the problem you are interested in b) structural equation modeling c) multiple regression d) relevant definitions and descriptions of unknown concepts.

relevant definitions and descriptions of unknown concepts

True experimental Design

research using an experimental group and a control group, to which test units are randomly assigned (pre-test/post test)

The strength of this approach to measuring dependent variables is ease and time of administration and does not require extensive training is _____. _____.a) ratings b) interviews c) self-report inventories d) physiological measures

self report inventories