Research and Program Evaluation

Single case investigations are often called

"idiographic studies" or "single-subject designs."

Note that statisticians have created nonparametric tests that parallel the popular parametric measures.

**

power of a statistical test

1 - beta (Power connotes a statistical test's ability to correctly reject a false null hypothesis.) power connotes a statistical test's ability to reject correctly a false null hypothesis

11 trends in counseling research

1. More studies seem to be sporting multiple authors and female authors. 2. Increased attention is being paid to multicultural issues. 3. Field-based professionals and practitioners are submitting fewer contributions. 4. Meta-studies are being used to summarize findings related to a given topic or a theme. Cohen's d effect size (ES) statistic is used to gauge how strong a relationship exists (i.e., small 0.2, medium 0.5, or large 0.8). 5. A majority of studies rely on graduate students and adults as subjects. 6. Qualitative (non-numerical) research popularized by luminaries such as Freud and Piaget seems to be making a comeback. Graduate programs are emphasizing qualitative procedures and research. 7. N = 1 single-subject designs seem to be making a comeback due to a number of advantages of this paradigm. First, only one person is required and counselors are very interested in individual change. The setting is usually a real-world situation rather than a laboratory setting. Finally, it is generally easier for consumers of mental health services to understand studies of this type since they generally need fewer complex statistical analyses. A type of single-subject (N = 1) numerical experiment using the ABABA design is common once again. This model, popularized in the 1970s by the behavior modification rage, tracks the client with an extended baseline, through treatment, to the outcome. Single-subject research is dubbed as idiographic, while studies using groups of individuals to discover general principles are called nomothetic. 8. Counselors and graduate students feel they need more training in APA publication guidelines in order to feel comfortable making journal submissions. 9. Kurt Lewin's concept of action research is popular because it is intended to improve the situation (not just advance knowledge) with local people/clients who will be better off at the end of the research. Self- surveys are often used to conduct Action Research. Action research bridges the gap between research and application/practice. 10. Using the Internet to conduct an experiment. Advantages can include rapid data collection, lower research costs, and very often the ability to secure very large sample sizes. 11. Neuroscience is being used to help guide diagnostic and treatment procedures. More females are being used in such studies since most traditional neuroscience findings are biased toward male humans and other mammals. The microbiome gut-mental health connection is emerging, especially in regard to bad bacteria and good bacteria (think probiotics such as organic yogurt or kefir) in the gastrointestinal tract.

minimum number of people for survey

100

minimum number of subjects for correlational research

30

parameter

A parameter is technically a value obtained from a population while a statistic is a value drawn from a sample. A parameter summarizes a characteristic of a population (e.g., the average male's height is 5'9"). * but don't we infer about population parameters from sample statistics?

percentiles vs. percentage scores

A percentage score is just another way of stating a raw score. A percentage score of 50 could be a very high, a very low, or an average score on the test. It merely says that the examinee got half of the answers correct. Percentile: A point on a ranking scale of 0 to 100. The 50th percentile is the midpoint; half the people in the population being studied rank higher and half rank lower. Graphically speaking, a distribution of percentile scores will always appear rectangular and flat.

between-subjects design

Different subjects for each condition **In a between-subjects design, each subject receives only one value of the IV.

true experiment

If your exam describes a true experiment—such as the biofeedback research described in the next several questions—except for the fact that the groups were not randomly assigned, then the new exams are calling this a causal comparative design. Expect to see this term on the exam. Data gleaned from the causal comparative ex post facto or after the fact design can be analyzed with a test of significance (e.g., a t test or ANOVA) just like any true experiment.

idiographic and nomothetic

Single-subject research is dubbed as idiographic, while studies using groups of individuals to discover general principles are called nomothetic.

what journal publishes more counseling research articles than any other periodical in our field?

The APA's Journal of Counseling Psychology

chi-square

The chi- square is a nonparametric statistical measure that tests whether a distribution differs significantly from an expected theoretical distribution.

Freud and research

The original case study methodology was popularized by Freud, though needless to say, unlike the behaviorists, Freud did not rely on numerical baseline measures.

biserial correlation

This merely indicates that one variable is continuous (i.e., measured using an interval scale) while the other is dichotomous. An example would be evident if you decided to correlate state licensing exam scores to NCC status (here the dichotomy is licensed/unlicensed). If both variables are dichotomous (i.e., two valued) then a phi-coefficient correlation is necessary.

threats to internal validity

Threats to internal validity include maturation of subjects (psychological and physical changes including fatigue due to the time involved), mortality (i.e., subjects withdrawing), instruments used to measure the behavior or trait, and statistical regression (i.e., the notion that extremely high or low scores would move toward the mean if the measure is utilized again).

Multivariate ANOVA (MANOVA)

When a study has more than one DV the term multivariate analysis of variance (MANOVA) is utilized.

phi-coefficient correlation

a correlation between two dichotomous variables; i.e., correlate NCC status with CCMHC status (has it/does not have it)

percentile rank

a descriptive statistic that tells the counselor what percentage of the cases fell below a certain level. Hence, if Joe's score puts him at the 50th percentile, then 50% of the people had raw scores lower than his particular score.

continuous variable

a quantitative variable that has an infinite number of possible values that are not countable

t-test

a statistical test used to evaluate the size and significance of the difference between two means

In the social sciences the accepted probability level is usually a. .05 or less. b. 1.0 or higher. c. .0001 or less. d. .05 or higher.

a. .05 or less. The two most popular levels of significance are .05 and .01. (a)

organismic variable

any relatively stable physical characteristic of an organism such as gender, eye color, height, weight, and body build, as well as such characteristics as intelligence, educational level, neuroticism, and prejudice.

A bimodal distribution has two modes (i.e., most frequently occurring scores). Graphically, this looks roughly like a. a symmetrical bell-shaped curve. b. a camel's back with two humps. c. the top half of a bowling ball. d. a mountain which is leaning toward the left.

b. a camel's back with two humps. When a curve exhibits more than two peaks it is known as a "multimodal" distribution. This can be contrasted to the curve with just a single peak (e.g., the normal curve) which is said to be "unimodal"

In a new study the clients do not know whether they are receiving an experimental treatment for depression or whether they are simply part of the control group. This is, nevertheless, known to the researcher. Thus, this is a a. double-blind study. b. single-blind study. c. baseline for an intensive N = 1 design. d. participant observer model.

b. single-blind study. In the single-blind study the subject would not know whether he or she is a member of the control group or the experimental group. This strategy helps eliminate "demand characteristics" which are cues or features of a study which suggest a desired outcome. In other words a subject can manipulate and confound an experiment by purposely trying to confirm or disprove the experimental hypothesis. Let us say that in the above-referenced experiment a subject is fond of the researcher. And let us further assume that a score on a standardized depression inventory will be used as the DV. Our subject might purposely answer the questions as if he is less depressed than he really is. A subject who disliked the researcher could present himself as even more depressed. Just in case you erroneously chose choice "c," please notice that the question used the word clients which is plural. N = 1 designs rely on a single individual for investigation purposes. Choice "d" describes a study in which the researcher actually participates in the study, while making observations about what transpired.

A professor of counselor education hypothesized that biofeedback training could reduce anxiety and improve the average score on written board exams. If this professor decides to conduct a formal experiment the IV will be the ________, and the DV will be the ________. a. professor; anxiety level b. anxiety level; board exam score c. biofeedback; board exam score d. board exam score; biofeedback

c. biofeedback; board exam score

correlational research

correlational research does not make use of the paradigm in which an IV is experimentally introduced.

Which level of significance would best rule out chance factors? a. .05 b. .01 c. .2 d. .001

d. .001 Some researchers refer to the level of significance as where "one draws the line" or the "cutoff point" between findings that should or should not be ascribed to chance factors. The significance level must be set low. If, for example, a researcher foolishly set the level at .50, then the odds would be 50-50 that the results were due to pure chance. I guarantee you a reputable journal would never touch an article with statistics like that! If you marked anything other than choice "d," you also should review question 720.

The researcher in question 727 now attempts a more complex experiment. One group receives no assertiveness training, a second group receives four assertiveness training sessions, and a third receives six sessions. The statistic of choice would be the a. mean. b. t test. c. two-way ANOVA. d. ANOVA.

d. ANOVA. This is a tough question. A one-way analysis of variance is used for testing one independent variable, while a two-way analysis of variance is used to test two independent variables. When a study has more than one DV the term multivariate analysis of variance (MANOVA) is utilized. The answer is choice "d" since the simple ANOVA or one-way analysis of variance is used when there is more than one level of a single IV, which in this case is the assertiveness training.

Experimenters should always abide by a code of ethics. The variable you manipulate/control in an experiment is the a. DV. b. dependent variable. c. the variable you will measure to determine the outcome. d. IV or independent variable.

d. IV or independent variable.

parametric tests

have more power than nonparametric statistical tests. Parametric tests are used only with interval and ratio data

repeated-measures comparison design

measures the same group of subjects without the IV and with the IV

descriptive statistics

merely describes data (e.g., the mean, the median, or the mode).

ANOVA

tests two or more groups while controlling for extraneous variables that are often called "covariates" The results of an ANOVA yield an F-statistic. The researcher then consults an F table for a critical value of F. If F obtained (i.e., computed) exceeds the critical F value in the table, then the null hypothesis is rejected.

two-way ANOVA

two-way analysis of variance is used to test two independent variables.

the Spearman correlation or Kendall's tau

used in place of the Pearson r when parametric assumptions cannot be utilized

Wilcoxon signed-rank test

used in place of the t test when the data are nonparametric and you wish to test whether two correlated means differ significantly (use the "co" to remind you of "correlated")

Kruskal-Wallis statistical test

used instead of the one-way ANOVA when the data are nonparametric

the Mann-Whitney U test

used to determine whether two uncorrelated means differ significantly when data are nonparametric (the "u" can remind you of "uncorrelated")

Causal Comparative Design

Causal Comparative design is a true experiment WITHOUT random assignment. Data from the causal comparative ex post factor 'after the fact' design can be analyzed with a test of significance, t test or ANOVA, just like any true experiment.

ex post facto study

Ex post facto literally means "after the fact," connoting a correlational study or research in which intact, preexisting groups are utilized. In the case of the ex post facto study, the IV was administered before the research began.

external validity

External validity, on the other hand, refers to whether the experimental research results can be generalized to larger populations (i.e., other people, settings or conditions). Thus, if the results of the study only apply to the population in the study itself then the external validity is said to be low.

inferential statistics

In order to compare two groups, "inferential statistics," which infer something about the population, are necessar

internal validity

Internal validity refers to whether the DVs were truly influenced by the experimental IVs or whether other factors had an impact.

ethical research

Now, in any experiment the counselor researcher is guided by ethics: this suggests first, that subjects are informed of any risks; second, that negative after-effects are removed; third, that you will allow subjects to withdraw at any time; fourth, that confidentiality of subjects will be protected; fifth, that the results of research reports will be presented in an accurate format that is not misleading; and sixth, that you will use only techniques that you are trained in. Research is considered a necessary factor for professionalism in counseling.

Assume the experiment in question 708 is conducted. The results indicate that the biofeedback helped raise written board exam scores but in reality this is not the case. The researcher has made a a. Type I error. b. Type II error. c. beta error. d. b and c.

a. Type I error. Questions like this can be very difficult. Be sure to utilize scratch paper to write down your thoughts if your exam allows you to do so! First, write down (or mentally picture) the null hypothesis regarding the experiment in question. In this case null would indicate that biofeedback did not raise board exam scores. This question tells you that the experimental results revealed that biofeedback did raise board scores, so you will reject the null hypothesis. The question then goes on to say that in reality the biofeedback did not really cause the results. Therefore, you have rejected null when it is true/applicable. This is the definition of a Type I or alpha error. Since the experimenter sets the alpha level, he or she is always cognizant of the probability of making a Type I error.

In a normal curve the mean, the median, and the mode all fall precisely in the middle of the curve. From a graphical standpoint the so-called normal or Gaussian curve (named after the astronomer/mathematician K. F. Gauss) looks like a. a symmetrical bell. b. the top half of a bowling ball. c. the top half of a hot dog. d. a mountain which is leaning toward the left.

a. a symmetrical bell. The normal curve is a theoretical notion often referred to as a "bell-shaped curve." The bell is symmetrical. Most physical and psychological traits are normally distributed. In other words, if enough data are collected in regard to a given trait, and a frequency polygon is constructed, it will resemble the bell-shaped curve. Curves that are not symmetrical (i.e., those which are asymmetrical) are called "skewed distributions."

A good guess would be that if you would correlate the length of CACREP graduates' baby toes with their CPCE scores the result would be a. close to 0.00. b. close to a perfect 1.00. c. close to a perfect negative correlation of -1.00. d. be about +.70.

a. close to 0.00. There is an absence of association here because as one variable changes the other variable varies randomly. The variation of one variable is most likely totally unrelated to the variation of the other.

P = .05 really means that a. differences truly exist; the experimenter will obtain the same results 95 times out of 100. b. differences truly exist; the experimenter will obtain the same results 99 times out of 100. c. there is a 95% error factor. d. there is a 10% error factor.

a. differences truly exist; the experimenter will obtain the same results 95 times out of 100.

A large study at a major university gave an experimental group of clients a new type of therapy that was intended to ameliorate test anxiety. The control group did not receive the new therapy. Neither the clients nor the researchers knew which students received the new treatment. This was a a. double-blind study. b. single-blind study. c. typical AB design. d. case of correlational research.

a. double-blind study. A double-blind study goes one step beyond the single-blind version by making certain that the experimenter is also unaware of the subjects' status. In fact, in the double-blind situation the persons assigned to rate or judge the subjects are often unaware of the hypothesis. This procedure helps eliminate confounding caused by "experimenter effects." Experimenter effects can flaw an experiment because the experimenter might unconsciously communicate his or her intent or expectations to the subjects.

In experimental terminology IV stands for ________ and DV stands for ________. a. independent variable; dependent variable b. dependent variable; independent variable c. individual variable; dependent variable d. independent variable, designer variable

a. independent variable; dependent variable Variables in an experiment are categorized as independent variables (IVs) or dependent variables (DVs). A variable is merely a behavior or a circumstance that can exist on at least two levels or conditions. In plain, simple, everyday English, a variable is a factor that "varies" or is capable of change. In an experiment the IV is the variable that the researcher manipulates, controls, alters, or wishes to experiment with. A neat little memory device is that IV begins with an "I," so imagine yourself as the researcher and remember "I manipulate the IV" or "I experiment with the IV." The DV expresses the outcome or the data. Here the memory device is a cinch: DV begins with a "D" and so does the word data. The DV expresses the data regarding factors you wish to measure. *IVs and DVs—the variables of the experimental trade—can be discrete (e.g., a brand of counseling or occupation) or continuous (e.g., height or weight).

Occam's Razor suggests that experimenters a. interpret the results in the simplest manner. b. interpret the results in the most complex manner. c. interpret the results using a correlation coefficient. d. interpret the results using a clinical interview.

a. interpret the results in the simplest manner. A word to the wise: Exams often refer to parsimony as Occam's razor, the principle of economy, or Lloyd Morgan's 1894 Canon (canon in this sense means "law"). Conway Lloyd Morgan was an English psychologist/physiologist, while William of Occam was a fourteenth-century philosopher and theologian. The early behaviorists (e.g., Watson) adhered closely to this principle. Key point: Have you ever placed a sticker on your car and tried to smooth it out? No matter how many times you attempt to do this the sticker usually retains a few trapped air bubbles. This analogy has often been used in conjunction with research, in the sense that flaws in research are often called bubbles.

Experiments emphasize parsimony, which means a. interpreting the results in the simplest way. b. interpreting the results in the most complex manner. c. interpreting the results using a correlation coefficient. d. interpreting the results using a clinical interview.

a. interpreting the results in the simplest way. Parsimonious literally means a tendency to be miserly and not overspend. A parsimonious individual is said to be overly economical and stingy. In research, we strive for parsimony in the sense that the easiest and less-complex explanation is said to be the best; an economical description if you will. Simply put, the simplest explanation of the findings is always preferred. The factor analysis mentioned in the previous answer is parsimonious in the sense that 10 tests which measure the dimensions of an effective counselor can be explained via a short measure which describes three underlying variables. Factor analysis, then, is concerned with data reduction.

If data indicate that students who study a lot get very high scores on state counselor licensing exams, then the correlation between study time and LPC exam scores would be a. positive. b. negative. c. 0.00. d. impossible to ascertain.

a. positive. A positive correlation is evident when both variables change in the same direction. A negative correlation is evident when the variables are inversely associated; one goes up and the other goes down. In the scenario for this question the relationship is positive since as study time increases, LPC exam scores also increase. A negative correlation (choice "b") would be expected when correlating an association like the number of dental cavities and time spent brushing one's teeth; as brushing time goes up dental cavities probably go down. Choice "c" or a zero correlation indicates an absence of a relationship between the variables in question.

The hunch is known as the experimental or alternative hypothesis. The experimental hypothesis suggests that a difference will be evident between the control group and the experimental group (i.e., the group receiving the IV). Thus, if the experiment in question 708 were conducted, the experimental hypothesis would suggest that a. the biofeedback would raise board scores. b. the control group will score better on the board exam. c. there will be no difference between the experimental and the control groups. d. the experiment has been confounded.

a. the biofeedback would raise board scores. An alternative hypothesis—which may be called the "affirmative hypothesis" on your exam—asserts that the IV has indeed caused a change.

The most common measures of central tendency are the mean, the median, and the mode. The mode is a. the most frequently occurring score and the least-important measure of central tendency. b. always 10% less than the mean. c. the arithmetic average. d. the middle score in the distribution of scores.

a. the most frequently occurring score and the least-important measure of central tendency. The mode is the highest or maximum point of concentration. The French phrase à la mode means "in style" or "in vogue." The mode is the score that is most "in style" or occurs the most. Just remember that pie à la mode has a "high concentration of calories." The modal score is the highest point on the curve. Hence, a test might tell you that a population of schizophrenics consists of 400 whites, 60 Asian Americans, and 100 African Americans and ask you to pick out the so-called modal category, rather than the modal score. In this case the highest value is held by the white population. Statisticians refer to choice "c" as the "mean" and choice "d" as the "median."

When a researcher uses correlation, then there is no direct manipulation of the IV. A researcher might ask, for example, how IQ correlates with the incidence of panic disorder. Again, nothing is manipulated; just measured. In cases such as this a correlation coefficient will reveal a. the relationship between IQ and panic disorder. b. the probability that a significant difference exists. c. an F test. d. percentile rank.

a. the relationship between IQ and panic disorder. A statistic that indicates the degree or magnitude of relationship between two variables is known as a "correlation coefficient" and is often abbreviated using a lower-case r. A coefficient of correlation makes a statement regarding the association of two variables and how a change in one is related to the change in another. Correlations range from 0.00, no relationship, to 1.0 or -1.0 which signify perfect relationships. Important: A positive correlation is not a stronger relationship than a negative one of the same numerical value. A correlation of -.70 is still indicative of a stronger relationship than a positive correlation of .60. The minus sign merely describes the fact that as one variable goes up the other goes down.

A counselor believes that clients who receive assertiveness training will ask more questions in counseling classes. An experimental group receives assertiveness training while a control group does not. In order to test for significant differences between the groups the counselor should utilize a. the student's t test. b. a correlation coefficient. c. a survey. d. an analysis of variance (ANOVA).

a. the student's t test. When comparing two sample groups the t test, which is a simplistic form of the analysis of variance, is utilized. The t test is used to ascertain whether two sample means are significantly different. The researcher sets the level of significance and then runs the experiment. The t test is computed and this yields a t value. The researcher then goes to a t table found in the index of most statistics' texts. If the t value obtained statistically is lower than the t value (sometimes called "critical t") in the table, then you accept the null hypothesis. Your computation must exceed the number cited in the table in order to reject null. If there are more than two groups, then the analysis of variance (choice "d") is utilized.

An experiment is said to be confounded when a. undesirable variables are not kept out of the experiment. b. undesirable variables are kept out of the experiment. c. basic research is used in place of applied research. d. the sample is random.

a. undesirable variables are not kept out of the experiment. I hope you didn't mark choices "b" and "d" since they are necessary for a proper experiment. Confounding is said to occur when an undesirable variable which is not controlled by the researcher is introduced in the experiment. Hint: Your exam could refer to this as a contaminating variable. Take a good hard look at choice "c." Basic research is conducted to advance our understanding of theory, while applied research (also called action research or experience-near research) is conducted to advance our knowledge of how theories, skills, and techniques can be used in terms of practical application. Often counselors assert that much of the research is not relevant to the actual counseling process and indeed they are correct.

experimental hypothesis

aka alternative hypothesis or affirmative hypothesis —asserts that the IV has indeed caused a change. (as opposed to null)

68-95-99.7 rule

almost all scores fall between 3 SDs of the mean

type I error

alpha

Type I error

alpha - rejects the null hypothesis when it is true The probability of committing a Type I error equals the level of significance mentioned earlier. Therefore, the level of significance is often referred to as the "alpha level."

P = .05 really means that a. five subjects were not included in the study. b. there is only a 5% chance that the difference between the control group and the experimental groups is due to chance factors. c. the level of significance is .01. d. no level of significance has been set.

b. there is only a 5% chance that the difference between the control group and the experimental groups is due to chance factors. Important note: Many experts in the field feel it is misleading when many exams still refer to this as the "95% confidence interval," meaning that the results would be due to chance only five times out of 100. When P = .05, differences in the experimental group and the control group are evident at the end of the experiment, and the odds are only one in 20 that this can be explained by chance. So once more for good measure (no pun intended!) your exam could refer to the "level of significance" as the level of confidence or simply the confidence level. The meaning is intended to be the same.

basic research vs. applied research

basic: quest for knowledge for its own sake. Applied: designed to solve specific problems. - aka action research or experience-near research

Type II error

beta - accept null when it is false The probability of committing a type II error is equal to one minus the power of the test

The study that would best rule out chance factors would have a significance level of P = a. .05. b. .01. c. .001. d. .08.

c. .001. The smaller the value for P the more stringent the level of significance. Here, the .001 level is the most stringent level listed, indicating that there is only one chance in 1,000 that the results are due to chance, versus one in 20 for .05, and one in 100 for .01. In plain, everyday English it is easier to get significant results using .08, .05, or .01, than it is using .001.

Which of the following would most likely yield a perfect correlation of 1.00? a. IQ and salary. b. ICD diagnosis and salary. c. Length in inches and length in centimeters. d. Height and weight.

c. Length in inches and length in centimeters. In the real world, correlations may be strong (e.g., choice "d"), yet they are rarely 1.00. Correlation is concerned with what statisticians call "covariation." When two variables vary together statisticians say the variables "covary positively," and when one increases while the other decreases they are said to "covary negatively."

In order for the professor of counselor education to conduct the experiment suggested in question 708 the experimental group would need to receive a. the manipulated IV. b. the biofeedback training. c. a and b. d. the organismic IV.

c. a and b. The experimental group receives the IV, which in this case is the biofeedback training. An organismic variable is one the researcher cannot control yet exists, such as height, weight, or gender. To determine whether an organismic IV exists you simply ask yourself if there is an experimental variable being examined which you cannot manipulate. In most cases, when you are confronted with IV/DV identification questions, the IV will be of the "manipulated variety."

To complete a t test you would consult a tabled value of t. In order to see if significant differences exist in an ANOVA you would consult a. the mode. b. a table for t values. c. a table for F values. d. the chi-square.

c. a table for F values. More elaborate tests (e.g., Tukey's, Duncan's multiple range, and Scheffe's test) can determine whether a significant difference exists between specific groups. Group comparison tests such as these are called "post hoc" or "a posteriori" tests for ANOVA calculations.

If a researcher changes the significance level from .05 to .001, then a. alpha and beta errors will increase. b. alpha errors increase but beta errors decrease. c. alpha errors decrease; however, beta errors increase. d. this will have no impact on Type I and Type II errors.

c. alpha errors decrease; however, beta errors increase. Let me mention as an aside that research can evoke fear, hostility, and anxiety in counselors. Why? Well, many counselors view research as cold, sterile, impersonal, and not related to the actual counseling process. There is also the fear of negative results related to the way one practices his or her craft.

The null hypothesis suggests that there will not be a significant difference between the experimental group which received the IV and the control group which did not. Thus, if the experiment in question 708 was conducted, the null hypothesis would suggest that a. all students receiving biofeedback training would score equally well on the board exam. b. systematic desensitization might work better than biofeedback. c. biofeedback will not improve the board exam scores. d. meta-analysis is required.

c. biofeedback will not improve the board exam scores. The null hypothesis asserts that the samples will not change (i.e., they will still be the same) even after the experimental variable is applied. Let me say that in a slightly different way: According to the null hypothesis the control group and the experimental group will not differ at the end of the experiment. The null hypothesis is simply that the IV does not affect the DV. Null means "nil" or "nothing." Null is a statement of "no difference." Choice "d" introduces the term meta-analysis, which is a study that analyzes the findings of numerous studies. Hence, a study of reality therapy that looked at the results of 20 reality therapy studies would be a meta-analysis.

In order for the professor of counselor education (see question 708) to conduct an experiment regarding his hypothesis he will need a(n) ________ and a(n) ________. a. biofeedback group; systematic desensitization group b. control group; systematic desensitization group c. control group; experimental group d. control group with at least 60 subjects; experimental group with at least 60 subjects

c. control group; experimental group The control group and the experimental group both have the same characteristics except that members of the control group will not have the experimental treatment applied to them. In this case, for example, the control group will not receive the biofeedback training. The control group does not receive the IV. The experimental group receives the IV. The basic presupposition is that the averages (or means) of the groups do not differ significantly at the beginning of the experiment. Choice "d" would also be a correct answer if it said 15 per group instead of 60. Remember that if you cannot randomly assign the subjects to the two groups then your exam will consider the research a quasi- experiment. Most experts suggest that you need at least 30 people to conduct a true experiment. Correlational research requires 30 subjects per variable while a survey should include at least 100 people.

A counselor educator is running an experiment to test a new form of counseling. Unbeknownst to the experimenter one of the clients in the study is secretly seeing a gestalt therapist. This experiment a. is parsimonious. b. is an example of Occam's Razor. c. is confounded/flawed. d. is valid and will most likely help the field of counseling.

c. is confounded/flawed. The experiment is said to be invalid (so much for choice "d") due to an extraneous independent variable (IV) (e.g., the gestalt therapy). Variables which are undesirable confound or "flaw" the experiment. The only experimental variable should be the IV—in this case the new form of counseling. The IV must have the effect on the dependent variable (here the DV would be some measure of the client's mental health). In this experiment any changes could not be attributed with any degree of certainty to the new form of counseling since DV changes could be due to the gestalt intervention (an extraneous word to the wise: Exams often refer to parsimony as Occam's Razor, the principle of economy, or confounding variable). All correlational research is said to be confounded.

When you see the letter P in relation to a test of significance it means a. portion. b. population parameter. c. probability. d. the researcher is using an ethnographic qualitative approach.

c. probability. The correct answer is choice "c" which refers to the probability or the level of significance. Traditionally, the probability in social science research (often indicated by a P) has been set at .05 or lower (i.e., .01 or .001). The .05 level indicates that differences would occur via chance only five times in 100. The significance level must be set before the experiment begins!

Behaviorists often utilize N = 1, which is called intensive experimental design. The first step in this approach would be to a. consult a random number table. b. decide on a nonparametric statistical test. c. take a baseline measure. d. compute the range.

c. take a baseline measure. "N," or the number of persons being studied, is one. This is a "case study" of one approach. This method is popular with behaviorists who seek overt (measurable) behavioral changes. The client's dysfunctional behavior is measured (this is called a baseline measure), the treatment is implemented, and then the behavior is measured once again (i.e., another baseline is computed). Exams sometimes delineate this paradigm using upper-case As and Bs and Cs such that As signify baselines, Bs intervention implementation, and Cs a second or alternative form of intervention. * Case studies are often misleading because the results are not necessarily generalizable.

The most valuable type of research is a. always conducted using a factor analysis. b. conducted using the chi-square. c. the experiment, used to discover cause-and-effect relationships. d. the quasi-experiment.

c. the experiment, used to discover cause-and-effect relationships. Experimental research is the process of gathering data in order to make evaluative comparisons regarding different situations. An experiment must have the conditions of treatment controlled via the experimenter and random assignments (also called randomization) used in the groups. An experiment attempts to eliminate all extraneous variables.

If the researcher in the previous question utilized two IVs then the statistic of choice would be the a. median. b. t test. c. two-way ANOVA or MANOVA. d. ANOVA.

c. two-way ANOVA or MANOVA. Two IVs requires a two-way ANOVA, three IVs, a three-way ANOVA, etc. (c)

A Type I error occurs when a. you have a beta error. b. you accept null when it is false. c. you reject null when it is true. d. you fail to use a test of significance.

c. you reject null when it is true. Okay, here it is: Time to plug in your handy dandy memory formula—"RA" or "reject when applicable/true." Since all statistical tests rely on probability there is always the possibility that the results were merely chance occurrences. Researchers call these chance factors "errors."

Hypothesis testing is most closely related to the work of a. Robert Hoppock. b. Sigmund Freud. c. Lloyd Morgan. d. R. A. Fisher

d. R. A. Fisher Hypothesis testing was pioneered by R. A. Fisher. A hypothesis is a hunch or an educated guess which can be tested utilizing the experimental model. A hypothesis might be that biofeedback raises board exam scores; or that reality therapy reduces dysfunctional classroom behavior in high school students; or perhaps that cognitive therapy relieves depression in males in the midst of a divorce. A hypothesis is a statement which can be tested regarding the relationship of the IV and the DV.

A Type II error a. is also called a beta error. b. means you reject null when it is applicable. c. means you accept null when it is false. d. a and c.

d. a and c. Although lowering the significance level (e.g., .01 to .001) lowers Type I errors, it "raises" the risk of committing a Type II or beta error. Simply think of the Type I/Type II relationship as a seesaw in the sense that when one goes up the other goes down. Hence, in determining an alpha level, the researcher needs to decide which error results in the most serious consequences. The safest bet is to set alpha at a very stringent level and then use a large sample size. If this can be accomplished, it is possible to make the correct decision (i.e., accept or reject null) the majority of the time.

From a purely statistical standpoint, in order to compare a control group (which does not receive the IV or experimental manipulation) to the experimental group the researcher will need a. a correlation coefficient b. only descriptive statistics. c. percentile rank. d. a test of significance.

d. a test of significance. The correct answer is that the researcher in this experiment will need a test of significance. Such statistical tests are used to determine whether a difference in the groups' scores is "significant" or just due to chance factors. In this case a t test would be used to determine if a significant difference between two means exists. This has been called the "two-groups" or "two-randomized-groups" research design. In this study, the two groups were independent of each other in the sense that the change (or lack of it) in one group did not influence the other group. Thus, this is known as an "independent group comparison design." If the researcher had measured the same group of subjects without the IV and with the IV, then the study would be a "repeated-measures comparison design." Exam hint: When a research study uses different subjects for each condition, some exams refer to the study as a "between- subjects design." If the same subjects are employed (e.g., such as in repeated measures) your exam could refer to it as a "within-subjects design."

Type I and Type II errors are called ________ and ________ respectively. a. beta; alpha b. .01; .05 c. a and b d. alpha; beta

d. alpha; beta A Type I (alpha error) occurs when a researcher rejects the null hypothesis when it is true; and a Type II error (beta error) occurs when you accept null when it is false. The memory device RA (as in "residence advisor") works well here so you can remember the principle as well as the sequence. Let "R" signify "reject when true" and "A"—which comes after "R"—signify "accept when false." If that memory device leaves you feeling apprehensive, here's another one using the "RA" abbreviation. Let "RA" be your first error (i.e., alpha, Type I) and remember this error occurs when you "R" (reject) null when you should "A" for accept it. Or better still use both "RA" devices. The probability of committing a Type I error equals the level of significance mentioned earlier. Therefore, the level of significance is often referred to as the "alpha level." 1 minus beta is called "the power of a statistical test." In this respect, power connotes a statistical test's ability to reject correctly a false null hypothesis.itting a Type I error equals the level of significance mentioned earlier. Therefore, the level of significance is often referred to as the "alpha level." 1 minus beta is called "the power of a statistical test." In this respect, power connotes a statistical test's ability to reject correctly a false null hypothesis. Parametric tests have more power than nonparametric statistical tests. Parametric tests are used only with interval and ratio data.

Dr. X discovered that the correlation between therapists who hold NCC status and therapists who practice systematic desensitization is .90. A student who perused Dr. X's research told his fellow students that Dr. X had discovered that attaining NCC status causes therapists to become behaviorally oriented. The student is incorrect because a. systematic desensitization is clearly not a behavioral strategy. b. this can only be determined via a histogram. c. the study suffers from longitudinal and maturational effects. d. correlation does not imply causal.

d. correlation does not imply causal. Shout it out: Correlation does not mean causal! Correlational research is quasi-experimental, and hence, it does not yield cause-effect data. A major research study, for example, might discover a very high correlation between the number of college students in a given geographical area and number of smart devices owned. Yet it would certainly be misleading to conclude that owning a lot of smart devices causes one to become a college student. Exam hint: When correlational data describe the nature of two variables, the term bivariate is utilized. If more than two variables are under scrutiny, then the term multivariate is used to describe the correlational paradigm.

Experimental is to cause and effect as correlational is to a. blind study. b. double-blind study. c. N = 1 design. d. degree of relationship.

d. degree of relationship. A correlation coefficient is a descriptive statistic which indicates the degree of "linear relationship" between two variables. Statisticians use the phrase "linear relationship" to indicate that when a perfect relationship exists (i.e., a correlation of 1.0 or -1.0) and it is graphed, a straight line is formed (see the Graphical Representations section of this book). The Pearson Product-Moment Correlation r is used for interval or ratio data while the Spearman rho correlation is used for ordinal data. Correlational research is not experimental and hence does not imply causality. So what do you do if your exam sneaks in a question regarding the type of data which must be used with the Pearson r versus the Spearman rho? I'd opt for a memory device. Pearson r, the most common correlation coefficient, uses I and R (interval and ratio data) as in "information and referral." Spearman rho ends in "o" as in ordinal.

Nondirective is to person-centered as a. psychological testing is to counseling. b. confounding is to experimenting. c. appraisal is to research. d. parsimony is to Occam's Razor.

d. parsimony is to Occam's Razor. A simple analogy question. Nondirective and person-centered therapy are synonymous; both refer to names given to Rogerian counseling. Parsimony is roughly synonymous with Occam's Razor. Important exam reminder: Most counselors see themselves as practitioners rather than researchers. Research, nevertheless, helps the entire field of counseling advance. It has been pointed out that we know about the work of many famous counselors and career counselors because of their published research not because of what transpired in their sessions.

A counselor educator decides to increase the sample size in her experiment. This will a. confound the experiment in nearly every case. b. raise the probability of Type I and Type II errors. c. have virtually no impact on Type I and Type II errors. d. reduce Type I and Type II errors.

d. reduce Type I and Type II errors. Raising the size of a sample helps to lower the risk of chance/error factors. Simply put: Differences revealed via large samples are more likely to be genuine than differences revealed using a small sample size.

chance factors

errors

ethnographic research

ethnographic research involves research that is collected via interviews, observations, and inspection of documents.

chi-square nonparametric test

examines whether obtained frequencies differ significantly from expected frequencies

factor analysis

refers to statistical procedures that use the important or underlying "factors" in an attempt to summarize a lot of variables. Hence, a test which measures a counselor's ability may try to describe the three most important variables (factors) that make an effective helper, although literally hundreds of factors may exist. Using factor analysis procedures, a brief test that measures the three major factors may be able to predict who will be an effective counselor as accurately as 10 other tests that examine hundreds of variables or so-called factors.

Cohen's d effect sizes - different strengths of relationships

small 0.2 medium 0.5 large 0.8

nonparametric statistical tests

statistical tests that are used with nominal and ordinal level data; also may be used with interval and ratio level data that are not normally distributed "distribution-free"

participant observer model

study in which the researcher actually participates in the study while making observations about what transpired

quasi-experiment

the researcher uses preexisting groups, and hence the IV (independent variable) cannot be altered (e.g., gender or ethnicity). In a quasi-experiment you cannot state with any degree of statistical confidence that the IV caused the DV (dependent variable). One popular type of quasi-experiment is known as the "ex post facto study."

within-subjects design

the same subjects are employed (e.g., such as in repeated measures) **In a within-subjects design, two or more values or levels of the IV are administered to each subject.

AB or ABA time-series design

the simplest type of single-subject research; popularized by behavior modifiers in 1960s and 70s; rely on continuous measurement; ranges from AB to ABAB; a baseline is secured (A), intervention is implemented (B), the outcome is examined via a new baseline (A), (B) intervention is implemented again In order to improve the research process, an ABAB design can be utilized to better rule out extraneous variables. If the pattern for the second AB administration mimics that of the first AB, then the chances increase that B (the intervention or so-called treatment) caused the changes rather than an extraneous variable. Some exams will refer to ABA or ABAB paradigms as "withdrawal designs." The rationale is that the behavior will move in the direction of the initial baseline each time the treatment is withdrawn if the treatment IV is responsible for the change. The ethical counselor must forego using a withdrawal or reversal design (as they are sometimes called) if the removal of the treatment variable could prove harmful to the subject or those who come in contact with the individual. Here, a simple AB or ABA must suffice. Remember that when a researcher employs more than one target behavior, the term multiple-baseline design probably will be used on your exam.

two-groups or two-randomized groups research design

the two groups were independent of each other in the sense that the change (or lack of it) in one group did not influence the other group. known as an independent group comparison design

discrete variables

variables that usually consist of whole number units or categories and are made up of chunks or units that are detached and distinct from one another

how many people to conduct a true experiment

~30 (15 in each group?)