NCE Prep, Ch. 8: Inferential Statistics

Steps in conducting a meta-analysis include the following:

-Establish criteria for including a study based on operational definitions (e.g., psychotherapy, counseling). -Locate empirical studies based on criteria. Include dis- sertations and theses as appropriate. -Consider and code variables that could be independent variables (e.g., length of treatment, counselor training, rigor of research design). -The dependent variable in a meta-analysis is the effect size (i.e., a measure of the strength of the relationship between two variables in a population) of the outcome. Calculate an effect size on any outcome variable in the study. Thus, there may be several effect sizes per study. • Effect sizes are compared and combined across studies and grouped according to independent variables of interest.

Multiple analysis of covariance (MANCOVA)

A MANCOVA is similar to an ANCOVA but involves multiple dependent variables.

Multiple analysis of variance (MANOVA)

A MANOVA is similar to an ANOVA, but involves multiple dependent variables.

Correlation Coefficient

A correlation coefficient provides information about the relationship between two variables. Correlations indicate three things: whether there is a relationship at all, the direction of that relationship, and the strength of the relationship. The stronger the relationship, the higher the absolute value of the correlation The valence (+ or -) that preceded the value indicates the direction of the relationship. A positive relationship involves variables that move in the same direction. For example, the relationship between height and weight is typically a positive relationship, in that as height increases, so does weight. A negative relationship indicates that, as one variable increases in value, the other decreases. For example, wealth and violent crime may have a negative relationship: as wealth increases, one's tendency to perpetuate violent crime decreases.

T-test

A t-test compares two means for one variable. Independent t-tests involve comparing two independent groups (with participants usually assigned randomly) on one dependent variable. An example of an independent t-test would be gender differences in achievement. Dependent t-tests (repeated measures t-tests) involve similar groups paired or matched in some meaningful way, or the same group tested twice. An example of a dependent t-test would be changes in test scores for the same group of high school students from a pretest to a posttest. T-tests provide a t ratio.

confirmatory factor analysis

Confirmatory factor analysis (CFA) refers to confirming the EFA results. The most common method is the maximum likelihood method (described above). After attaining a factor solution, one tests how the overall model fits the data using a fit index. Some fit indices are chi-square statistic, Tucker-Lewis Index, comparative fit index, standardized root mean-squared residual, and root mean squared error of approximation.

Wilcoxon's signed ranks test

Equivalent to a dependent t-test. This test involves ranking the amount and direction of change for each pair of scores. For example, this test would be appropriate for assessing changes in perceived level of competency before and after a training program.

Inferential Statistics

Inferential statistics is a statistical model generally used to try to describe results beyond what is garnered from the data alone. In other words, the inferential model attempts to infer from the sample data conclusions about a population of interest. Inferential statistics are distinct from descriptive statistics, which simply describe the data Inferential statistics allow a researcher to make assessments about populations based on the probability of particular differences without having to test every person within the population. For example, a study of voter positions during an election year would rely on some form of inferential statistics to describe the population, unless a researcher intends to ask all voters their opinion.

what is important to note about correlations?

It is important to note that correlations only indicate relationships between variables and do not indicate causation. For example, we can say that X is related to Y, but we cannot say X causes Y or Y causes X. In addition, other variables may mediate or moderate the relationship between X and Y.

Nonparametric Statistics

Nonparametric statistics are used when professional counselors are only able to make a few assumptions about the distribution of scores in the underlying population. They are suggested when nominal or ordinal data are involved or when interval or ratio data are not distributed normally (i.e., are skewed).

parametric statistics

Parametric statistics are used when statistical assumptions are met.

Regression studies

Prediction studies are extensions of correlational studies and are known as regression studies. If professional counselors know that two variables have a high correlation, they have opportunities to predict outcomes (although we cannot also explain outcomes as we can in experimental designs).

Friedman's rank test

Similar to Wilcoxon's signed- ranks test in that it is designed for repeated measures. In addition, it may be used with more than two comparison groups.

Kolmogrov-Smrinov Z Procedure

Similar to the Mann-Whitney U test but more appropriate to use when samples are smaller than 25 participants.

Several examples of parametric statistics

T-Test Analysis of Variance (ANOVA) Post-hoc analysis Factorial ANOVA Analysis of covariance (ANCOVA) Multiple analysis of variance (MANOVA) Multiple analysis of covariance (MANCOVA)

Factor analysis

The purpose of a factor analysis is to reduce a larger number of variables (often items on an assessment) to a smaller number of factors (groups or factors). Factors are hypothetical constructs that explain covariation among variables; each factor explains a certain percentage of variance (i.e., strength of the association between variables and a factor).

The two forms of factor analysis?

The two forms of factor analysis are exploratory factor analysis and confirmatory factor analysis. Exploratory factor analysis (EFA) involves an initial examination of potential models (or factor structures) that best categorize the variables. EFA involves two steps: (a) extraction of factors and (b) rotation and interpretation of those factors. These two steps can be thought of as analogous to mining for precious metals. Let's say one is mining a forest comprising several precious metals along with various other materials, such as stones, debris, rocks, and plants (variables or items). One further believes that various metals (gold, silver) are present and could be categorized as precious metals (factor) in that they all share something in common (common variance) even though each has unique aspects (unique variance). Factor extraction would consist of pulling out those various metals to clump them as precious metals, and factor rotation would consist of cleaning them up to see them better in order to ensure they are, in fact, precious metals. Three common types of factor extractions are (a) princi- pal axis factoring (i.e., analysis of the common variance among variables or items, also referred to as communal- ity); (b) principal components analysis (i.e., analysis of both the common and the unique variance of variables to find as many linear combinations [principal components] as possible to explain as much information as possible about the variables); and (c) maximum likelihood method (i.e., likelihood that a correlation matrix representing the variables is similar to that derived from a population correlation matrix). Factor rotation involves changing the reference point for variables for easier interpretation. Rotation does not change the relationship among variables, just the perspective. The decision of what type of rotation to use is based on theoretical notions regarding whether or not the factors are correlated or uncorrelated. The two types of factor rotation are orthogonal (when factors are uncorrelated) and oblique (when factors are correlated).

Inferential Statistics rely on what?

nferential statistics rely on the use of statistical tests. The choice of a statistical test involves several factors, including one's research question, the types of groups one is using (e.g., independent, dependent, repeated measures groups), the number of IVs and DVs in one's study, the scale of measurement and the ability to meet several statistical assumptions, including the following: • Data for the dependent variable(s) are approximately normally distributed. • Samples were randomly selected and/or assigned .• An interval or ratio scale of measurement was used for each of the variables involved in the study.

Correlations are often presented with...

scatterplots

What are the three types of regression?

• Bivariate regression. How well scores from an inde- pendent variable (predictor variable) predict scores on the dependent variable (criterion variable). • Multiple regression. Involves more than one predictor variable; each predictor variable is weighted (beta weights) in a regression equation to determine the contri- bution of each variable to the criterion variable. Gener- ally, the more predictor variables, the stronger the prediction that can be made. • Logistic regression. Dependent variable is dichotomous. This form of regression may be similar to a bivariate or multiple regression.

correlation values range from ?

-1.00 to +1.00 and are typically represented by the Pearson product moment correlation coefficient (commonly referred to as "Pear- son r"). A correlation of 1.00 indicates a perfect posi- tive relationship, and a 1.00 indicates a perfect negative relationship. Other types of correlation coeffi- cients are "Spearman r" (for comparing rank-order vari- ables), biserial correlation coefficients (comparing one continuous and one artificially dichotomous or dummy coded variable), and point biserial correlation coeffi- cients (relating one continuous and one true dichoto- mous variable).

Examples of nonparametric statistics

Chi-square test Mann-Whitney U Test Kolmogorov-Smirnov Z Procedure Kruskal-Wallis test Wilcoxon's signed-ranks test Friedman's rank test

Meta-analysis

Although the results of a single study are often helpful, similar studies often yield disparate or even contradictory results. Which results do you believe? Can the results be combined somehow to indicate what would happen on average? A meta- analysis allows a researcher to combine and synthesize the results of numerous similar studies for particular outcome or dependent variables. Examples of outcome variables might include depression symptoms, grade point averages, high school dropout rates, and so forth. Meta-analysis has been popular in counseling for several decades due to a desire to show what works in counseling, for whom, and under what conditions. Two seminal meta-analyses in psychotherapy (i.e., Eysenck, 1952; Smith & Glass, 1977) have sparked interest in determining the effectiveness of counseling (or "psychotherapy," as they labeled it) and discussing limitations of conducting meta-analyses in counseling in general

Analysis of variance (ANOVA) and Post hoc analysis

An ANOVA involves having at least one independent variable in a study with three or more groups or levels. For example, an independent variable such as household income (with categories of "Below $20,000," "$20,001-$40,000." "$40,001-$60,000," and "Above $60,000") would have four groups or levels. An ANOVA is an extension of the t-test used to minimize Type I error (i.e., rejecting a null hypothesis when it is true); it provides an F ratio, which tells one if two or more of the group means are statistically different. Post hoc analysis allows one to examine every possible pairing of groups for a particular independent variable after one has concluded there are main effects (i.e., significant difference among two or more groups making up a single independent variable).

Degrees of Freedom

An important concept often noted in inferential statistics is degrees of freedom. Degrees of freedom (df) refers to the number of scores, or categories of a variable, that are "free to vary." The df value is important in most inferential statistics formulas, and computing df depends on the statistical test used. It is equal to the number of scores or variable categories minus the number of parameters (generally represented as n - 1). Typically, the more degrees of freedom, the greater the certainty that the sample is representative of the population. A good way to think of degrees of freedom is as a restrictor. For example, if one has three variable cat- egories that add up to 20 (a + b + c = 20), the first two values may be random numbers, but the third is restricted. Thus, the df in this case would be 3 - 1 = 2.

Mann-Whitney U Test

Analogous to a parametric independent t-test, except the Mann-Whitney U test uses ordinal data instead of interval or ratio data. This test compares the ranks from two groups. For example, a professional counselor might use this test to compare students in Grades 9 through 12 with education aspiration (i.e., high school, 2-year degree, 4-year degree, graduate degree) as a dependent variable.

Krusskal-Wallis Test

Analogous to an ANOVA. This test is an extension of the Mann-Whitney U test when there are three or more groups per independent variable.

Analysis of Covariance (ANCOVA)

Analysis of covariance (ANCOVA). This test includes an independent variable as a covariate, or a variable that needs to be statistically adjusted and controlled in order to look at the relationship of other independent variables and the dependent variable. (If one is also interested in the independent variable designated as a covariate, one would simply conduct a factorial ANOVA instead.) An example of an ANCOVA might be examining the relationship between household income and work satisfaction, with gender as a covariate—that is, the statistical effects of gender are removed from the analysis to control for any effects gender might have on work satisfaction.

Chi-square test

Used with two or more categorical or nominal variables, where each variable contains at least two categories. All scores must be independent—that is, the same person cannot be in multiple categories of the same variable. The professional counselor forms the categories and then counts the frequency of observations or categories. Then, the reported (or observed) frequencies are compared statistically with theoretical or expected frequencies. An example of a study suitable for this statistic might be investigating the relationship between the decision to terminate counseling (yes, no) and the gender of the professional counselor (male, female). A chi-square would test whether the tallies for the decision to quit counseling by gender of counselor are significantly different from those expected in the population.

Spurious Correlation

When a correlation overrepresents or underrepresents the actual relationship, it is referred to as a spurious correlation. Overestimation occurs when a common variable is part of both the independent variable and dependent varia- ble (overlap) or when a third unmeasured variable is common to both variables (such as age for reading achievement and body weight). Correlationsmaybemisleadingifmeasuresareunreli- able in that they may show a lower relationship between the two variables (known as attenuation). Restrictionofrange,orhavingasamplethatisnotrep- resentative of the population (either too heterogeneous or too homogenous), will result in inaccurate relation- ships. Itiseasytoconfusethedecimalsofcorrelationsasper- centages. We square correlation values to understand the percent of variance shared among variables (i.e., coefficient of determination, r2).

Factorial ANOVA

When there is more than one independent variable, a facto- rial ANOVA is used. Factorial ANOVAs yield both main effects and interaction effects (i.e., significant differences among groups across two or more independent variables)—for example, if two treatments (e.g., cognitive- behavioral therapy [CBT] and interpersonal therapy [IPT]) are compared for effectiveness on males and females and different treatments were significantly more effective with different genders—for example, CBT worked significantly better for males than females while IPT worked significantly better for females than males). Post hoc analysis would determine the existence and direction of these interactions.